Tumblelog by Soup.io
Newer posts are loading.
You are at the newest post.
Click here to check if anything new just came in.

May 19 2018

How To Deploy Django Channels To Production

In this article, we will see how to deploy django channels to production and how we can scale it to handle more load. We will be using nginx as proxy server, daphne as ASGI server, gunicorn as WSGI server and redis for channel back-end.

Daphne can serve HTTP requests as well as WebSocket requests. For stability and performance, we will use uwsgi/gunicorn to serve HTTP requests and daphne to serve websocket requests.

We will be using systemd to create and manage processes instead of depending on third party process managers like supervisor or circus. We will be using ansible for managing deployments. If you don't want to use ansible, you can just replace template variables in the following files with actual values.

Nginx Setup

Nginx will be routing requests to WSGI server and ASGI server based on URL. Here is nginx configuration for server.

server {
    listen {{ server_name }}:80;
    server_name {{ server_name }} www.{{ server_name }};

    return 301 https://avilpage.com$request_uri;

server {
    listen {{ server_name }}:443 ssl;
    server_name {{ server_name }} www.{{ server_name }};

    ssl_certificate     /root/certs/avilpage.com.chain.crt;
    ssl_certificate_key /root/certs/avilpage.com.key;

    access_log /var/log/nginx/avilpage.com.access.log;
    error_log /var/log/nginx/avilpage.com.error.log;

    location / {
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header Host $http_host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_redirect off;

    location /ws/ {
            proxy_http_version 1.1;

            proxy_read_timeout 86400;
            proxy_redirect     off;

            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection "upgrade";
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Host $server_name;

    location /static {
        alias {{ project_root }}/static;

    location  /favicon.ico {
        alias {{ project_root }}//static/img/favicon.ico;

    location  /robots.txt {
        alias {{ project_root }}/static/txt/robots.txt;


WSGI Server Setup

We will use gunicorn for wsgi server. We can run gunicorn with

$ gunicorn avilpage.wsgi --bind --log-level error --log-file=- --settings avilpage.production_settings

We can create a systemd unit file to make it as a service.


WorkingDirectory={{ project_root }}
Environment="DJANGO_SETTINGS_MODULE={{ project_name }}.production_settings"
ExecStart={{ venv_bin }}/gunicorn {{ project_name}}.wsgi --bind --log-level error --log-file=- --workers 5 --preload

ExecReload=/bin/kill -s HUP $MAINPID
ExecStop=/bin/kill -s TERM $MAINPID


Whenever server restarts, systemd will automatically start gunicorn service. We can also restart gunicorn manually with

$ sudo service gunicorn restart

ASGI Server Setup

We will use daphne for ASGI server and it can be started with

$ daphne avilpage.asgi:application --bind --port 9000 --verbosity 1

We can create a systemd unit file like the previous one to create a service.

Description=daphne daemon

WorkingDirectory={{ project_root }}
Environment="DJANGO_SETTINGS_MODULE={{ project_name }}.production_settings"
ExecStart={{ venv_bin }}/daphne --bind --port 9000 --verbosity 0 {{project_name}}.asgi:application
ExecReload=/bin/kill -s HUP $MAINPID
ExecStop=/bin/kill -s TERM $MAINPID



Here is an ansible playbook which is used to deploy these config files to our server. To run the playbook on server avilpage.com, execute

$ ansible-playbook -i avilpage.com, django_setup.yml


Now that we have deployed channels to production, we can do performance test to see how our server performs under load.

For WebSockets, we can use Thor to run performance test.

thor -C 100 -A 1000 wss://avilpage.com/ws/books/

Our server is able to handle 100 requests per second with a latency of 800ms. This is good enough for low traffic website.

To improve performance, we can use unix sockets instead of rip/port for gunicorn and daphne. Also, daphne has support for multiprocessing using shared file descriptors. Unfortunately, it doesn't work as expected. As mentioned here, we can use systemd templates and spawn multiple daphne process.

An alternate way is to use uvicorn to start multiple workers. Install uvicorn using pip

$ pip install uvicorn

Start uvicorn ASGI server with

$ uvicorn avilpage.asgi --log-level critical --workers 4

This will spin up 4 workers which should be able to handle more load. If this performance is not sufficient, we have to setup a load balancer and spin up multiple servers(just like scaling any other web application).

May 18 2018

DRF - Optimizing ModelViewSet queries

Using Django REST Framework for Model views there is always the issue of making duplicated queries without either prefetching the objects that will be accessed using the serializer and as such will lead to large number of queries being made to the database.

This will help in optimizing the queryset ...

May 15 2018

Django Channels 2.0 to Production Environment

Learn how to deploy a Django C...

New Course: Creating and Distributing Python Packages

Myself and Audrey Roy Greenfeld have released a course, Creating and Distributing Python Packages. Features it includes:

  • Covers Python packaging details including setup.py, not to mention automating things like testing, versioning, documentation, dependency management, and checking for security issues.
  • Available in English or Spanish.
  • Demonstrates Cookiecutter and Cookiecutter PyPackage.
  • Will help fund our open source work in a way that doesn't involve us directly asking people for money (because we suck at that).

Who is this course good for?

Do you have a script, function, class, or snippet of code you copy/paste from project to project? If that's the case, this course is for you. We'll teach you how to package that up and add automated tests, documentation, and so much more.

This especially applies to organizations that share code between projects. If you are copy/pasting common code that means updating it across projects is challenging and error prone, introducing the risk of bugs and security concerns. Plus, odds are it's not tested reliably. By packaging your copy/pasting code and adding automation, it becomes just another dependency, easily updated across multiple projects.

Or what if you have an API that needs a Python client SDK? If that's the case, then this course is perfect for you. You can use this course as a foundation for how to build a tool that isn't just installable on your machine, but is tested across multiple operating systems and versions of Python.

Funding open source

Here's a confession: We're terrible at asking money for Cookiecutter and its ecosystem. While popular amongst individuals and organizations, Cookiecutter is critically underfunded and that's hurting its progress. In fact, as I write this our monthly funding (Raphael Pierzina, me) for Cookiecutter is at $53/month). That means we're almost unpaid volunteers, meaning most Cookiecutter work is done using our personal time instead of work time. Honestly, that's unsustainable for a project of this complexity used by so many people on so many different operating systems with different versions of Puthon.

By taking this course you will help fund us to continue working on Cookiecutter and related projects. Since we believe in paying it forward, if we get enough students, we plan on donating to the Python Package Index (PyPI).

Why you should take this course

  • You want to learn how to package your python code.
  • You want to automate testing, versioning, testing, documentation, dependency management, and checking for security issues.
  • You or your company uses Cookiecutter or related projects and want to support them financially.

May 11 2018

Always return namespaces in Django REST Framework

By default, when you hook up a model to Django REST Framework and run a query in JSON format, what you get is a list. E.g.

For GET localhost:8000/api/mymodel/

  {"id": 1, "name": "Foo"},
  {"id": 2, "name": "Bar"},
  {"id": 3, "name": "Baz"}

This isn't great because there's no good way to include other auxiliary data points that are relevant to this query. In Elasticsearch you get something like this:

  "took": 106,
  "timed_out": false,
  "_shards": {},
  "hits": {
    "total": 0,
    "hits": [],
    "max_score": 1

Another key is that perhaps today you can't think of any immediate reason why you want to include some additonal meta data about the query, but perhaps some day you will.

The way to solve this in Django REST Framework is to override the list function in your Viewset classes.


# views.py
# views.py
from rest_framework import viewsets

class BlogpostViewSet(viewsets.ModelViewSet):
    queryset = Blogpost.objects.all().order_by('date')
    serializer_class = serializers.BlogpostSerializer


# views.py
from rest_framework import viewsets

class BlogpostViewSet(viewsets.ModelViewSet):
    queryset = Blogpost.objects.all().order_by('date')
    serializer_class = serializers.BlogpostSerializer

    def list(self, request, *args, **kwargs):
        response = super().list(request, *args, **kwargs)
        # Where the magic happens!
        return response

Now, to re-wrap that, the response.data is a OrderedDict which you can change. Here's one way to do it:

# views.py
from rest_framework import viewsets

class BlogpostViewSet(viewsets.ModelViewSet):
    queryset = Blogpost.objects.all().order_by('date')
    serializer_class = serializers.BlogpostSerializer

    def list(self, request, *args, **kwargs):
        response = super().list(request, *args, **kwargs)
        response.data = {
            'hits': response.data,
        return response

And if you want to do the same the "detail API" where you retrieve a single model instance, you can add an override to the retrieve method:

def retrieve(self, request, *args, **kwargs):
    response = super().retrieve(request, *args, **kwargs)
    response.data = {
        'hit': response.data,
    return response

That's it. Perhaps it's personal preference but if you, like me, prefers this style, this is how you do it. I like namespacing things instead of dealing with raw lists.

"Namespaces are one honking great idea -- let's do more of those!"

From import this

Note! This works equally when you enable pagination. Enabling pagination immediately changes the main result from a list to a dictionary. I.e. Instead of...

  {"id": 1, "name": "Foo"},
  {"id": 2, "name": "Bar"},
  {"id": 3, "name": "Baz"}

you now get...

  "count": 3,
  "next": null,
  "previous": null,
  "items": [
    {"id": 1, "name": "Foo"},
    {"id": 2, "name": "Bar"},
    {"id": 3, "name": "Baz"}

So if you apply the "trick" mentioned in this blog post you end up with...:

  "hits": {
    "count": 3,
    "next": null,
    "previous": null,
    "items": [
      {"id": 1, "name": "Foo"},
      {"id": 2, "name": "Bar"},
      {"id": 3, "name": "Baz"}

May 10 2018

May 07 2018

Creating Dynamic Forms with Django

What is a dynamic form and why would you want one?

Usually, you know what a form is going to look like when you build it. You know how many fields it has, what types they are, and how they’re going to be laid out on the page. Most forms you create in a web app are fixed and static, except for the data within the fields.

A dynamic form doesn’t always have a fixed number of fields and you don’t know them when you build the form. The user might be adding multiple lines to a form, or even multiple complex parts like a series of dates for an event. These are forms that need to change the number of fields they have at runtime, and they’re harder to build. But the process of making them can be pretty straightforward if you use Django’s form system properly.

Django does have a formsets feature to handle multiple forms combined on one page, but that isn’t always a great match and they can be difficult to use at times. We’re going to look at a more straightforward approach here.

Creating a dynamic form

For our examples, we’re going to let the user create a profile including a number of interests listed. They can add any number of interests, and we’ll make sure they don’t repeat themselves by verifying there are no duplicates. They’ll be able to add new ones, remove old ones, and rename the interests they’ve already added to tell other users of the site about themselves.

Start with the basic static profile form.

class Profile(models.Model):
    first_name = models.CharField()
    last_name = models.CharField()
    interest = models.CharField()

class ProfileForm(forms.ModelForm):
    first_name = forms.CharField(required=True)
    last_name = forms.CharField(required=True)
    interest = forms.CharField(required=True)

class Meta:
    model = Profile

Create a fixed number of interest fields for the user to enter.

class Profile(models.Model):
    first_name = forms.CharField()
    last_name = forms.CharField()

Class ProfileInterest(models.Model):
    profile = models.ForeignKey(Profile)
    interest = models.CharField()

Class ProfileForm(forms.ModelForm):
    first_name = forms.CharField(required=True)
    last_name = forms.CharField(required=True)
    interest_0 = forms.CharField(required=True)
    interest_1 = forms.CharField(required=True)
    interest_2 = forms.CharField(required=True)

    def save(self):
        Profile = self.instance
        Profile.first_name = self.cleaned_data[“first_name”]
        Profile.last_name = self.cleaned_data[“last_name”]

        For i in range(3):
           interest = self.cleaned_data[“interest_{}”.format(i]
               profile=profile, interest=interest)

But since our model can handle any number of interests, we want our form to do so as well.

Class ProfileForm(forms.ModelForm):
    first_name = forms.CharField(required=True)
    last_name = forms.CharField(required=True)

    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        interests = ProfileInterest.objects.filter(
        for i in range(len(interests) + 1):
            field_name = 'interest_%s' % (i,)
            self.fields[field_name] = forms.CharField(required=False)
                self.initial[field_name] = interests[i].interest
            Except IndexError:
                self.initial[field_name] = “”
        field_name = 'interest_%s' % (i + 1,)
        self.fields[field_name] = forms.CharField(required=False)
        self.fields[field_name] = “”

    def clean(self):
        interests = set()
        i = 0
        field_name = 'interest_%s' % (i,)
        while self.cleaned_data.get(field_name):
           interest = self.cleaned_data[field_name]
           if interest in interests:
               self.add_error(field_name, 'Duplicate')
           i += 1
           field_name = 'interest_%s' % (i,)
       self.cleaned_data[“interests”] = interests

    def save(self):
        profile = self.instance
        profile.first_name = self.cleaned_data[“first_name”]
        profile.last_name = self.cleaned_data[“last_name”]

        for interest in self.cleaned_data[“interests”]:

Rendering the dynamic fields together

You won’t know how many fields you have when rendering your template now. So how do you render a dynamic form?

def get_interest_fields(self):
    for field_name in self.fields:
        if field_name.startswith(‘interest_’):
            yield self[field_name]

The last line is the most important. Looking up the field by name on the form object itself (using bracket syntax) will give you bound form fields, which you need to render the fields associated with the form and any current data.

{% for interest_field in form.get_interest_fields %}
    {{ interest_field }}
{% endfor %}

Reducing round trips to the server

It’s great that the user can add any number of interests to their profile now, but kind of tedious that we make them save the form for every one they add. We can improve the form in a final step by making it as dynamic on the client-side as our server-side.

We can also let the user enter many more entries at one time. We can remove the inputs from entries they’re deleting, too. Both changes make this form much easier to use on top of the existing functionality.

Adding fields on the fly

To add fields spontaneously, clone the current field when it gets used, appending a new one to the end of your list of inputs.

$('.interest-list-new').on('input', function() {
    let $this = $(this)
    let $clone = $this.clone()

You’ll need to increment the numbering in the name, so the new field has the next correct number in the list of inputs.

    let name = $clone.attr('name')
    let n = parseInt(name.split('_')[1]) + 1
    name = 'interest_' + n

The cloned field needs to be cleared and renamed, and the event listeners for this whole behavior rewired to the clone instead of the original last field in the list.

    $clone.attr('name', name)
    $this.off('input', arguments.callee)
    $clone.on('input', arguments.callee)

Removing fields on the fly

Simply hide empty fields when the user leaves them, so they still submit but don’t show to the user. On submit, handle them the same but only use those which were initially filled.

    .on(“blur”, function() {
        var value = $(this).val();
        if (value === “”) {

Why dynamic forms matter

An unsatisfying user experience that takes up valuable time may convince users to leave your site and go somewhere else. Using dynamic forms can be a great way to improve user experiences through response time to keep your users engaged.

Setup React

Below is a reference we made t...

May 06 2018

Staging Django for Production & Local Development

## How do you have different s...

Angular 6|5 Tutorial (with RESTful Django Back-End)

Throughout this Angular 6 tutorial, we'll learn to build a web application with Angular 6, the latest version of Angular--the most popular framework/platform for building mobile and desktop client side applications created and used internally by Google.

By the end of this Angular 6 tutorial, you'll learn:

  • how to install the latest version of Angular CLI,
  • how to use the Angular 6 CLI to generate a new Angular 6 project,
  • how to use Angular 6 to build a simple CRM application,
  • what's a component and component-based architecture
  • how to create Angular 6 components,
  • how to add component routing and navigation,
  • how to use HttpClient to consume a REST API etc.

The Django Back-End

We'll make use of a simple CRM API built with Django and Django REST framework. Since this is an Angular tutorial we'll not focus on building the API as this will be the subject of a separate tutorial but you can grab the source code of the back-end API from this repository

You can use the following commands to start the development server:

# Clone the project and navigate into it
$ git clone https://github.com/techiediaries/django-crm
$ cd django-crm

# Create a virtual environment and install packages
$ pipenv install

# Activate the virtual environment
$ pipenv shell 

# Create and migrate the database then run the local development server
$ python manage.py migrate
$ python manage.py runserver

You server will be running from http://localhost:8000.

We are using pipenv, the officially recommended package management tool for Python so you'll need to have it installed. The process is quite simple depending on your operating system.

Installing the Angular CLI (v6.0.0)

Make sure you have Node.js installed, next run the following command in your terminal to install Angular CLI v 6.0.0.

npm -g install @angular/cli

You can check the installed version by running the following command:

ng version

This is the output I'm getting:

    / \   _ __   __ _ _   _| | __ _ _ __     / ___| |   |_ _|
   / △ \ | '_ \ / _` | | | | |/ _` | '__|   | |   | |    | |
  / ___ \| | | | (_| | |_| | | (_| | |      | |___| |___ | |
 /_/   \_\_| |_|\__, |\__,_|_|\__,_|_|       \____|_____|___|

Angular CLI: 6.0.0
Node: 8.11.1
OS: linux x64

Package                      Version
@angular-devkit/architect    0.6.0
@angular-devkit/core         0.6.0
@angular-devkit/schematics   0.6.0
@schematics/angular          0.6.0
@schematics/update           0.6.0
rxjs                         6.1.0
typescript                   2.7.2

Now, you're ready to create a project using Angular CLI v6. Simply run the following command in your terminal:

ng new crmapp

The CLI will automatically generate a bunch of files common to most Angular 6 projects and install the required dependencies for your project.

We will mostly be working inside the src/app folder. This is the directory structure of the project:

You can serve your application locally by running the following commands:

# Navigate inside your project's folder
$ cd crmapp

# Serve your application
$ ng serve

You application will be running from http://localhost:4200.

This is a screen-shot of home page of the application:

Components in Angular 6|5|4

Now what's a component?

A component is a TypeScript class with an HTML template and an optional set of CSS styles that control a part of the screen.

Components are the most important concept in Angular 6. An Angular 6 application is basically a tree of components with a root component (the famous AppComponent). The root component is the one contained in the bootstrap array in the main NgModule module app.module.ts.

One important aspect of components is re-usability. A component can be re-used throughout the application and even in other applications. Common and repeatable code that performs a certain task can be encapsulated into a re-usable component that can be called whenever we need the functionality it provides.

Each bootstrapped component is the base of its own tree of components. Inserting a bootstrapped component usually triggers a cascade of component creations that fill out that tree. source

Component-Based Architecture

An Angular application is made of several components forming a tree structure with parent and child components.

A component is an independent block of a big system (web application) that communicates with the other building blocks (components) of the system using inputs and outputs. A component has associated view, data and behavior and may have parent and child components.

Components allow maximum re-usability, easy testing, maintenance and separation of concerns.

Let's now see this practically. Head over to your Angular application project folder and open the src/app folder. You will find the following files:

  • app.component.css: the CSS file for the component
  • app.component.html: the HTML view for the component
  • app.component.spec.ts: the unit tests or spec file for the component
  • app.component.ts: the component code (data and behavior)
  • app.module.ts: the application main module

Except for the last file which contains the declaration of the application main (root) Module, all these files are used to create a component. It's the AppComponent: The root component of our application. All other components we are going to create next will be direct or un-direct children of the root component.

Demystifying the AppComponent (The Root Component of Angular Applications)

Go ahead and open the src/app/app.component.ts file and let's understand the code behind the main/root component of the application.

First, this is the code:

import { Component } from  '@angular/core';
    selector:  'app-root',
    templateUrl:  './app.component.html',
    styleUrls: ['./app.component.css']
export  class  AppComponent {
    title  =  'app';

We first import the Component decorator from @angular/core then we use it to decorate the TypeScript class AppComponent. The Component decorator takes an object with many parameters such as:

  • selector: specifies the tag that can be used to call this component in HTML templates just like the standard HTML tags
  • templateUrl: indicates the path of the HTML template that will be used to display this component (you can also use the template parameter to include the template inline as a string)
  • styleUrls: specifies an array of URLs for CSS style-sheets for the component

The export keyword is used to export the component so that it can be imported from other components and modules in the application.

The title variable is a member variable that holds the string 'app'. There is nothing special about this variable and it's not a part of the canonical definition of an Angular component.

Now let's see the corresponding template for this component. If you open src/app/app.component.html this is what you'll find:

<div  style="text-align:center">
Welcome to !
    <img  width="300"  alt="Angular Logo"  src="data:image/svg+xml;....">

    <h2>Here are some links to help you start: </h2>
    <h2><a  target="_blank"  rel="noopener"  href="https://angular.io/tutorial">Tour of Heroes</a></h2>
    <h2><a  target="_blank"  rel="noopener"  href="https://github.com/angular/angular-cli/wiki">CLI Documentation</a></h2>
    <h2><a  target="_blank"  rel="noopener"  href="https://blog.angular.io/">Angular blog</a></h2>

The template is a normal HTML file (almost all HTML tags are valid to be used inside Angular templates except for some tags such as <script>, <html> and <body> etc.) with the exception that it can contain template variables (in this case the title variable) or expressions ({{...}}) that can be used to insert values in the DOM dynamically. This is called interpolation or data binding. You can find more information about templates from the docs.

You can also use other components directly inside Angular templates (via the selector property) just like normal HTML.

If you are familiar with the MVC (Model View Controller) pattern, the component class plays the role of the Controller and the HTML template plays the role of the View.

Angular Components in Action

After getting the theory behind Angular components, let's now create the components for our simple CRM application.

Our REST API, built with Django, exposes these endpoints:

  • /api/accounts: create or read a paginated list of accounts
  • /api/accounts/<id>: read, update or delete an account

  • /api/contacts: create or read a paginated list of contacts

  • /api/contacts/<id>: read, update or delete a contact

  • /api/leads: create or read a paginated list of leads

  • /api/leads/<id>: read, update or delete a lead

  • /api/opportunities: create or read a paginated list of opportunities

  • /api/opportunities/<id>: read, update or delete an opportunity

Before adding routing to our application we first need to create the application's components so based on the exposed REST API architecture we can initially divide our application into these components:

  • AccountListComponent: this component displays and controls a tabular list of accounts
  • AccountCreateComponent: this component displays and controls a form for creating or updating accounts

  • ContactListComponent: displays a table of contacts

  • ContactCreateComponent: displays a form to create or update a contact

  • LeadListComponent: displays a table of leads

  • LeadCreateComponent: displays a form to create or update a lead

  • OpportunityListComponent: displays a table of opportunities

  • OpportunityCreateComponent: displays a form to create or update an opportunity

Let's use the Angular CLI to create the components

ng generate component AccountList
ng generate component AccountCreate

ng generate component ContactList
ng generate component ContactCreate

ng generate component LeadList
ng generate component LeadCreate

ng generate component OpportunityList
ng generate component OpportunityCreate

This is the output of the first command:

CREATE src/app/account-list/account-list.component.css (0 bytes)
CREATE src/app/account-list/account-list.component.html (31 bytes)
CREATE src/app/account-list/account-list.component.spec.ts (664 bytes)
CREATE src/app/account-list/account-list.component.ts (292 bytes)
UPDATE src/app/app.module.ts (418 bytes)

You can see that the command generates all the files to define a component and also updates src/app/app.module.ts.

If you open src/app/app.module.ts after running all commands, you can see that all components are automatically added to the AppModule declarations array.:

import { BrowserModule } from  '@angular/platform-browser';

import { NgModule } from  '@angular/core';

import { AppComponent } from  './app.component';

import { AccountListComponent } from  './account-list/account-list.component';

import { AccountCreateComponent } from  './account-create/account-create.component';

import { ContactListComponent } from  './contact-list/contact-list.component';

import { ContactCreateComponent } from  './contact-create/contact-create.component';

import { LeadListComponent } from  './lead-list/lead-list.component';

import { LeadCreateComponent } from  './lead-create/lead-create.component';

import { OpportunityListComponent } from  './opportunity-list/opportunity-list.component';

import { OpportunityCreateComponent } from  './opportunity-create/opportunity-create.component';


declarations: [
imports: [
providers: [],
bootstrap: [AppComponent]
export  class  AppModule { }

If you are creating components manually, you need to make sure to include manually so they can be recognized as part of the module.

Adding Angular Routing

Angular CLI provides the --routing switch (ng new crmapp --routing) that enables you to add routing automatically but we're going to add routing manually for the sake of understanding the various pieces involved in adding component routing to your Angular application.

In fact, adding routing is quite simple:

  • add a separate module (which can be called AppRoutingModule) in a file app-routing.module.ts, and import the module by including it in the imports of main AppModule,
  • add <router-outlet></router-outlet> in app.component.html (this is where the Angular Router will insert components matching the current path),
  • add routes (each route is an object with properties such as path and component etc.).

This is the initial content of app-routing.module.ts:

import { NgModule } from '@angular/core';
import { Routes, RouterModule } from '@angular/router';

const routes: Routes = [

  imports: [RouterModule.forRoot(routes)],
  exports: [RouterModule]
export class AppRoutingModule { }

The routes will contain all the routes of the application. After creating the components we'll see how to add routes to this array.

For now, we want to redirect the visitor to the /accounts path when the home URL is visited so the first path we'll add is:


The pathMatch specifies the matching strategy. full means that we want to fully match the path.

Next let's add the other paths:

{ path:  '', redirectTo:  'accounts', pathMatch:  'full' },
    path:  'accounts',
    component:  AccountListComponent
    path:  'create-account',
    component:  AccountCreateComponent
    path:  'contacts',
    component:  ContactListComponent
    path:  'create-contact',
    component:  ContactCreateComponent
    path:  'leads',
    component:  LeadListComponent
    path:  'create-lead',
    component:  LeadCreateComponent
    path:  'opportunities',
    component:  OpportunityListComponent
    path:  'create-opportunity',
    component:  OpportunityCreateComponent


Now open src/app/app.module.ts and import the routing module then add it to the imports array:

import {AppRoutingModule} from  './app-routing.module';


declarations: [


imports: [
export  class  AppModule { }

Finally, open src/app/app.component.html then add the navigation links and the router outlet:

<a [routerLink]="'/accounts'"> Accounts </a>
<a [routerLink]="'/create-account'"> Create Account </a>

<a [routerLink]="'/contacts'"> Contacts </a>
<a [routerLink]="'/create-contact'"> Create Contact </a>

<a [routerLink]="'/leads'"> Leads </a>
<a [routerLink]="'/create-lead'"> Create Lead </a>

<a [routerLink]="'/opportunities'"> Opportunities </a>
<a [routerLink]="'/create-opportunity'"> Create Opportunity </a>


Consuming the REST API Using Angular 6 HttpClient

Now that we've created the different components and added routing and navigation, let's see how to use the HttpClient of Angular 6 to consume the RESTful API back-end.

First, you need to add the HttpClientModule module to the imports array of the main application module

import { HttpClientModule } from  '@angular/common/http';


declarations: [

imports: [


providers: [],
bootstrap: [AppComponent]


export  class  AppModule { }

Create an Angular 6 Service/Provider

A service is a global class that can be injected in any component. It's used to encapsulate code that can be common between multiple components in one place instead of repeating it throughout various components.

Now, lets create a service that encapsulates all the code needed for interacting with the REST API. Using Angulat CLI run the following command:

ng service api

Two files: src/app/api.service.ts and src/app/api.service.spec.ts will be generated. The first contains code for the service and the second contains tests.

Open src/app/api.service.ts then import and inject the HttpClient class.

import { Injectable } from  '@angular/core';
import { HttpClient} from  '@angular/common/http';

providedIn:  'root'

export  class  APIService {

    constructor(private  httpClient:  HttpClient) {}


Angular 6 provides a way to register services/providers directly in the @Injectable() decorator by using the new providedIn attribute. This attribute accepts any module of your application or 'root' for the main app module. Now you don't have to include your service in the providers array of your module.

Getting Contacts/Sending HTTP GET Request

Let's start with the contacts API endpoint.

  • First we'll add a method to consume this endpoint in our global API service,
  • next we'll inject the API service and call the method from the corresponding component class (ContactListComponent)
  • and finally we'll display the result (the list of contacts) in the component template.

Open src/app/api.service.ts and add the following method:

export  class  APIService {
API_URL  =  'http://localhost:8000';
constructor(private  httpClient:  HttpClient) {}
    return  this.httpClient.get(`${this.API_URL}/contacts`);

Next, open src/app/contact-list/contact-list.component.ts and inject the APIService then call the getContacts() method:

import { Component, OnInit } from  '@angular/core';
import { APIService } from  '../api.service';

    selector:  'app-contact-list',
    templateUrl:  './contact-list.component.html',
    styleUrls: ['./contact-list.component.css']

export  class  ContactListComponent  implements  OnInit {

private  contacts:  Array<object> = [];
constructor(private  apiService:  APIService) { }
ngOnInit() {
public  getContacts(){
    this.apiService.getContacts().subscribe((data:  Array<object>) => {
        this.contacts  =  data;

Now let's display the contacts in the template. Open src/app/contact-list/contact-list.component.html and add the following code:

My Contacts
<table  style="width:100%">
    <th>First Name</th>
    <th>Last Name</th>
<tr *ngFor="let contact of contacts">
    <td>  </td>
    <td>  </td>
    <td>  </td>
    <td>  </td>
    <td>  </td>


This is a screen-shot of the component:

Creating Contacts/Sending HTTP POST Request

Now let's create a method to send HTTP Post request to create a random contact. Open the API service file and add the following method:

    return  this.httpClient.post(`${this.API_URL}/contacts/`,contact);

Next let's call this method from the ContactCreateComponent to create a contact. First open src/app/contact-create/contact-create.component.ts and add the following code:

import { Component, OnInit } from  '@angular/core';
import { APIService } from  '../api.service';


selector:  'app-contact-create',

templateUrl:  './contact-create.component.html',

styleUrls: ['./contact-create.component.css']


export  class  ContactCreateComponent  implements  OnInit {
constructor(private  apiService:  APIService) { }

ngOnInit() {}


var  contact  = {
    account:  1,
    address:  "Home N 333 Apartment 300",
    createdBy:  1,
    description:  "This is the third contact",
    email:  "abbess@email.com",
    first_name:  "kaya",
    isActive: true,
    last_name: "Abbes",
    phone: "00121212101"
this.apiService.createContact(contact).subscribe((response) => {

For now, we're simply hard-coding the contact info for the sake of simplicity.

Next open src/app/contact-create/contact-create.component.html and add a button to call the method to create a contact:

Create Contact
<button (click)="createContact()">
    Create Contact


Throught this Angular 6 tutorial, we've seen how to use different Angular concepts to create simple full-stack application with Angular and Django. You can find the source code in this repository.

May 05 2018

May 04 2018

Overriding Field Widgets In Django Doesn't Work. Template Not Found. The Solution

One day you may want to override a default Django form input widget. Let's say; you'll want to tweak a Date widget, so it has type="date" on it.

So you'd go out and do the regular drill:

1) Find the source file inside the Django dir ...

Read now

May 02 2018

django-translated-fields – localized model fields without magic

django-translated-fields – localized model fields without magic

There are many ways to save and retrieve multilingual content in a database; countless blog posts, emails and software packages have been written discussing or helping with this problem.

Two main approaches exist to tackle the problem:

  1. Use a table for the language-independent content, and a table for language-specific content. The latter most often has a foreign key to the former and a language field. There will be a record in the latter table for each record in the former, or less if some dataset isn’t available in all languages. django-hvad, django-parler and also FeinCMS 1’s translations module follow this approach.
  2. Use only one table, but use several fields to store the localized data. django-modeltranslation is probably the best known app implementing this approach.

(Other ways of course exist. Among the more interesting packages (to me) are django-nece using Postgres’ JSONB fields and django-vinaigrette using gettext.)

Why write another package?

The features they provide are at costly to implement and hard to maintain. For example, django-modeltranslation supports adding translations to third party apps which themselves do not support any translations, but to do this it has to not only provide properties for attribute access on models, but also hook into querying, into form generation, into model admin classes, and implement generic fallback logic etc. so that the current language is respected everywhere transparently.

This does not only sound complex, it is! And the efforts and ingenuity that went into supporting those features have to be respected – I certainly do.

But, couldn’t I just help out instead of adding another package solving the same problem?. Yes, that I could. But time is limited, and even taking future maintenance into account it sometimes is easier – and more fun – to rewrite than to refactor, especially if you’re not trying to solve the exact same problem.


And that’s where django-translated-fields enters the fray.

While other packages contain thousands of lines of code, this package contains a good-enough solution in less than 50 lines of code, when ignoring the translated_attributes class decorator which is orthogonal to the TranslatedField class. Lines of code is not the only interesting metric of course, but it is a very good predictor of maintenance cost.

Of course the package has significantly less features than any of the other packages mentioned above, but it hits the sweet spot where its features are sufficient for most of our projects.

And living in a country with four official languages (and english isn’t even one of those) it should be easy to believe that after a few years you’ll have plenty of experience providing users and customers with ways to work with multilingual content, and knowing what is necessary and what isn’t.

May 01 2018

April 30 2018

Support disabling options in a select widget

Provide the ability to disable a select option with Django2. This can be done during the widget initialization (i.e when the widget field is created) or during form initialization.

This will disable the select option based on a context or by specifying the values that should be disabled.

April 27 2018

April 25 2018

RESTful APIs and Django - Emad Mokhtar

(Summary of a talk at the Amsterdam python meetup)

Ehmad Mokhtar used Django a lot. And he loves web APIs: you can have many intergrations. You build one back-end and it can be used by many other services and front-ends. It is also core for microservices and "service oriented architectures".

So: APIs = business! Without APIs, you have no intergrations. So often you won't have as much business.

What is a RESTful API? It is an architectural style that uses HTTP: you work on URLs ("resources") and you use HTTP verbs ("GET", "DELETE", "POST").

If you build an API, developers are your API customers. So you need, for instance, great documentation. You need to keep that in mind all the time.

If you use django and you want to build an API: use django rest framework (DRF). DRF feels like django itself. It uses the ORM. And it is perhaps even better documented than Django itself.

A tip he has is to use two views per model. In django, you can have a separate list view, a delete view, a regular view, etc. In DRF you are best off having two:

  • One for listing items and creating new ones. (/customers/)
    • GET: list of customers.
    • POST: add a new one.
  • One for showing one item, editing it, deleting it. (/customers/:id/)
    • GET: customer details.
    • PUT: update customer info.
    • PATCH: partially update customer.
    • DELETE: delete customer.

You could use a generic single ModelViewSet, but Emad prefers having control of his URLs.

A tip: don't go too deep with your hierarchy.

Another tip: version your API from the beginning. Prefix your URLs with /v1/, /v2/, for instance. This reduces breakage.

And: always return the coorect HTTP status codes. Look at https://httpstatuses.com/ . If you create a customer, return a 201 CREATED. If you request an unknown customer: 404 NOT FOUND.

(Note by Reinout: look at https://http.cat/, too :-) )

http://ccbv.co.uk/ is a great resource for django's class based views. http://www.cdrf.co/ is the same for DRF.


  • Don't use http basic auth. You force the customer to send his password all the time. That is bad for an API.
  • Django's session authentication: use it only internally.
  • It is much better to use TokenAuthentication (an external package).
  • Also good: JWT (json web token) authentication. There's a plug-in for django rest framework.

For permissions:

  • User permissions: "is admin?", "is authenticated?".
  • Regular model permissions.
  • Object permissions: django-guardian is a third-party django app.

Handy: there is support for throttling:

  • AnonRateThrottle for anonymous users.
  • UserRateThrottle: throttling per authenticated user.
  • ScopeRateThrottle for specific API scopes. You can set that per API endpoint.

When testing, use DRF test classes. APIRequestFactory, APIClient, APISimpleTestCase, etcetera. They're specially made to make your tests easier.

For functional tests, you can use vcrpy to make mocking your web API much faster. You use it to give it a sort of test fixture instead of having it call the real API.

Another tip: use the 'postman' app to play with your API and to test it out in detail. He demoed the API: nice!

File checksums in Python: the hard way - Shane Kerr

(Summary of a talk at the Amsterdam python meetup)

Shane is data hoarder. He hates losing data. He has lots of data. He could move it to the cloud, but he doesn't trust that: cloud companies go away after a time. And: disks are cheap now, so he doesn't need to.

But: bad things happen to good data. Disks go corrupt. You can use checksums to detect problems.

In the ideal work, the os/hardware detects problems and everything just works. But it isn't true for linux RAID (which doesn't do checks). ext4 doesn't do checking at all. ZFS could work, but the license is deliberately linux-GPL-unfriendly.

So: he wants proper checksums. You could use sha1sum in a shell script and pipe the checksums to a file. But the simple case doesn't handle metadata. And it isn't parallel.

So... he started a python program. Python is "batteries included": it has a great standard library. For instance os.walk() which goes through all the directories and files. With os.path.join() and os.path.normpath() you get usable full filepaths. With hashlib.sha224() he could get a hash per file.

He tried it out with various python versions:

  • python3 (cpython3) is the base.
  • python 2: works, but for new programs he wouldn't bother supporting it. Python 2 is end-of-life in two years' time.
  • pypy: normally much faster, but not in this case.
  • jython: worked, with a few small changes.
  • ironpython: didn't work. Missing libraries.

Issue: localization of file names. The problem is that file systems don't have language settings. Linux ext4 is often UTF-8, but it isn't mandatory. NTFS and VFAT are often UTF-16.

Python 3's standard libraries try to be smart. If you ask for files in b'/some/dir', you get byte strings. With '/some/dir', you get strings (unicode), unless there's something that isn't properly encodable.

There are also problems with timestamps: FAT (still used on USB sticks!) has a two-second-resolution. Modern OSs have a nanosecond resolution. Python by default doesn't use a float type that has enough resolution for that. There are ways around it.

Which algorithms?

  • Checksums?

  • CRC?

  • Hash function?

  • Cryoptographic hashing? md5: workable, but old. Would be fine for this purpose, but it would mean a great deal of explaining (because there have been attacks on it).

    He uses SHA-224 (SHA-2). Today he would have used BLAKE2.

Now the multiprocessing model of the program. They splitted it into the major CPU-bound workloads:

  • Main thread: os.walk, executing the 'stat' calls (for metadata). IO bound.
  • Worker threads (calculate hash values).
  • Serializer thread (outputs values in the correct order).

Actually, he uses multiprocessing, not threading`. So it runs multiple processes instead of threads, which avoids Python's GIL ("global interpreter lock").

The program is at https://github.com/shane-kerr/fileinfo

Two years ago he did a fun experiment by using python's tar module: https://github.com/shane-kerr/fv . Tar already calculates checksums and as long as you don't actually put the file contents into the tarfile, but instead inject a short comment about the file...

The origins of Python: the ABC language - Rolf Zwart

(Summary of a talk at the Amsterdam python meetup)

How to compare python to ABC? ABC is the unsuccessful language that was the inspiration for python. Guido van Rossum was originally the main implementor of the ABC language.

ABC was intended for the casual user. Home users. But also experts in other fields, but that were not programming experts. The focus was on friendliness of the language and on programmer productivity, not program efficiency.

It also was used as a teaching/research language.

Research on ABC was started before 1980. The first version was released in 1987. In the end of 1989, Guido started developing python, with the 0.9.0 version being released to Usenet in february 1991. His research group needed an easy-to-use scripting language for the distributed OS they were working on, that was why he started building Python. (He chose the name because he was a fan of the English TV series monty python's flying circus).

ABC was, like python, developed at the CWI (centrum voor wiskunde en informatica) in the Netherlands. The claim of ABC was that the programs build with it were typically around a quarter of the size of the equivalent Pascal or C programs. Some key features:

  • Statement grouping by indentation. Now you know where python got that from!
  • Only two basic data types.
  • Strong dynamic typing.
  • It had an interactive interpreter (nice: it also uses the >>> prompt).

A syntax example:

>>> WRITE 2**1000
>>> PUT 1/(2*1000) in x
>>> WRITE 1 + 1/x

In python, functions sometimes return something, sometimes not. In ABC, they were different:

>>> HOW TO MAKE fun WITH x:
        function body

        RETURN ~22/7

Some similarities and features inspired by ABC:

  • Dynamic typing, no type declarations.
  • High level data structures (lists, dicts) that can be combined.
  • Interactive interpreter, >>> prompt.
  • Focus on programmer efficiency.
  • All types have a canonical representation (like __repr__).
  • Readability counts: the syntax should be clean.
  • Grouping by indentation.
  • A block is introduced by a colon (:).
  • Slicing in strings.
  • Automatic garbage collection.
  • The C implementation compiles to byte code. It is interpreted by a high level VM.

ABC also had its problems and deficiencies:

  • It perhaps was ahead of its time.

  • It was too strict, too pure. It tried too hard to be perfect.

  • It wanted to stand on its own, in a closed world, so no interface with the outside world except text via stdin/stdout. Even no file I/O.

  • No interaction with external devices.

    He had to work on interaction with a graphical terminal. To get that to work, he had to hack escape codes into his unix shell to get commands from the closed ABC program to the external terminal...

  • One single namespace (instead of a separate namespace per function/module/etc, for instance, like in python).

The strong points of python:

  • It adopted the well-designed, high level elements of ABC...
  • ... but it was pragmatic. It dropped the fundamentalist harness and jargon from ABC.
  • Terminology and notation is familiar from C-like languages.
  • It took good ideas from Smalltalk (OO), modula-3 (modules, exceptions).
  • Interfaces via the OS. No problems with that.
  • Files are OK, of course.
  • Guido released it as open source (well, that term didn't exist yet, but ok).

A quote by Tim Peters on Python: in many ways, it's a dull language, borrowing solid old concepts from many other languages & styles: boring syntax, unsurprising semantics, few automatic coercions, etc etc. But that's one of the things I like about it.

Should you want to look at ABC: https://homepages.cwi.nl/~steven/abc/

Older posts are this way If this message doesn't go away, click anywhere on the page to continue loading posts.
Could not load more posts
Maybe Soup is currently being updated? I'll try again automatically in a few seconds...
Just a second, loading more posts...
You've reached the end.

Don't be the product, buy the product!