Tumblelog by Soup.io
Newer posts are loading.
You are at the newest post.
Click here to check if anything new just came in.

May 30 2018

Best React Books 2018

List of current React/JavaScript books.

May 29 2018

Angular 6|5 Tutorial: Integrating Angular with Django

In the previous Angular 6 tutorial we've seen how to build a CRUD web application with a Django REST framework API back-end. In this tutorial we'll see how we can integrate the Angular 6 front-end with the Django back-end.

After creating both the back-end and front-end apps we need to integrate them i.e instead of taking the approach where both applications are completely separated we'll serve the front-end application using a Django view. In development we'll have both Django development server and Angular/Webpack dev server running but for production we'll only need a Django server.

To use this approach you need to tweak the Webpack settings of your front-end project, use the webpack-bundle-tracker (from npm) and use the django-webpack-loader package (from PyPI)

The webpack-bundle-tracker is a Webpack plugin that generates a stats file containing meta data information about the assets of your front-end application generated by Webpack.

We'll start by installing the webpack-bundle-tracker module then update the Webpack configuration file to make use of this plugin.

npm install webpack-bundle-tracker --save

Next you need to eject the Webpack configuration from the Angular 6 CLI using

ng eject

If the ejection is successful you'll find a webpack.config.js in the root of your folder.

Open webpack.config.js and import BundleTracker from webpack-bundle-tracker then locate the plugins entry and add the following code

var BundleTracker = require('webpack-bundle-tracker');

module.exports  = {
        new BundleTracker({filename: '../webpack-stats.json'})

Next add the publicPath setting

"output": {
    "path":  path.join(process.cwd(), "dist"),
    "filename":  "[name].bundle.js",
    "chunkFilename":  "[id].chunk.js",
    "crossOriginLoading":  false,

"devServer": {
    "historyApiFallback":  true,
    "publicPath":  "",//2

If you serve your application you'll have a ../webpack-stats.json in the parent folder i.e the root of the Django project.

After ejecting your Webpack configuration from the Angular 6 CLI you won't be able to use ng serve instead you'll have to use npm run start to serve your application.

This is a screenshot of a project where you can see the webpack.config.js file in the front-end application and a generated webpack-stats.json in the root folder of the project

Next let's install The django-webpack-loader package which will take care of reading the webpack-stats.json and insert the assets in a Django template.

Head back to your terminal the run the following command:

pip install django-webpack-loader

In your settings.py file add webpack_loader to the list of installed apps:


Then add this configuration object:

    'DEFAULT': {
        'BUNDLE_DIR_NAME': '',

        'STATS_FILE': os.path.join(BASE_DIR, 'webpack-stats.json'),

You can find more settings that you can use via this link.

Serving the Angular 6 Application

Now let's create the view to serve the Angular 6 application. Open core/views and add the following view function:

from django.shortcuts import render

def  home(request):
    return render(request, 'core/home.html')

Next you need to create the home.html template so create a templates/core folder inside the core application then add a home.html with the following content:

{% load render_bundle from webpack_loader %}

<html  lang="en">
<meta  charset="UTF-8">
<base  href="/">
<title>A Simple CRM with Django and Angular 6</title>


{% render_bundle 'inline' %}
{% render_bundle 'polyfills' %}
{% render_bundle 'styles' %}
{% render_bundle 'vendor' %}
{% render_bundle 'main' %}


Now you need to add the URL for the home page in urls.py:

from django.contrib import admin
from django.urls import path
from django.conf.urls import url
from core import views as coreviews

urlpatterns =[


path('admin/', admin.site.urls)

That's it. You should now be able to see your Angular 6 page when visiting the Django web application

Fixing Hot Code Reload

If you change the source code of your front-end application you will not get updates hot code reloaded without manually refreshing the page if you are navigating your application from That means HCR is not working properly so simply open webpack.config.js and add the following setting:

"devServer": {
    "historyApiFallback":  true,
    "publicPath":  "",//2,
    "headers": {
        'Access-Control-Allow-Origin':  '\\*'//3

That's because http://localhost:8000 sends requests to the Webpack dev server (http://localhost:4200) to get source code changes so we need to update headers to allow request from all origins.


Throughout this tutorial we have integrated both Angular 6 front-end and the Django REST API back-end.

May 28 2018

QuerySet Filters on Many-to-many Relations

May 25 2018

Djangocon: friday lightning talks

(One of my summaries of a talk at the 2018 European djangocon.)

The stenographers - Sheryll and Andrew

The stenographers are the ones that provide LIVE speech to text subtitles for the talks. Wow.

They use "shorthand machines" for steno. It is like a piano where you press multiple keys to form words. On wednesday the speakers talked 46000 words...

How fast do people talk? Anything between 180 and 250, though 300 also occurs. The handiest are speakers that speak in a regular tempo. And not too fast.

They ask presenters for notes and texts beforehand. That helps the software pick the right words.

Pytest-picked - Ana Paula Gomes

Say you have a codebase that has tests that take a long time. You've just changed a few files and don't want to run the full test before the commit. You could run git status and figure out the files to test.

But you can do it automatically with pytest-picked.

It is a small plugin, but she wants to improve it with better guessing and also with support for testing what changed on a branch.

How to build a treehouse - Harry Biddle

A talk about building actual treehouses!

One of the craziest is "Horace's cathedral", a huge one that was closed by the fire brigade for fire risks...

If you're going to do it, be nice to your tree. Look at "garnier bolt" for securing your tree house.

Give the tree room to grow.

Recommended book: "the man who climbs trees".

What about DRF's renderers and parsers - Martin Angelov

They started with just plain django. Then they added DRF (django rest framework). Plus react.

Then it became a single page app with django, drf, react, redux, sagas, routers, etc. (Well, there not quite there yet).

Python and javascript are different. Python uses snake_case, javascript uses camelCase. What comes out of django rest framework also often is snake_case, which looks weird in javascript.

They tried to fix it.

  • JS utils. On-the-fly translation. Hard when debugging.
  • React middleware.
  • Then he used a custom CamelCase renderer in django rest framework :-)

Git Menorahs considered harmful - Shai Berger

A bit of Jewish tradition: a menorah is a 7-armed candle holder (a temple menorah) or 9 armed (hanukkah menorah).

What do we use version control for?

  • To coordinate collaborative work.
  • To keep a record of history.

But why do we need the history? To fix mistakes. Figure out when something was broken. Reconsider decisions. Sometimes you need to revert changes.

What do we NOT need in the history? The actual work process. Commits for typo fixes, for instance.

A git menorah is a git repo with many branches and changes. Bad. Use git rebase to reshape the history of your work before pushing. Follow the example of the django git repository.

5 minutes for your mental health - Ducky

What is really important for your mental health?

  • Sleep!
  • Exercise.

As weird as it sounds, you can 'feel' in your body how you're feeling. There are some exercises you can do. Close your eyes, put your mind in your fingertips. Slowly move inside your body. Etc. "Body scan".

Meta programming system, never have syntax errors again - Ilja Bauer

MPS is a system for creating new custom languages. It has a "projectional editor".

A normal editor shows the code, which is converted to an AST (abstract syntax tree) and it gets byte-compiled.

A projectional editor has the AST as the basis and "projects" it as .py/java/whatever files purely as output. The advantage is that renaming stuff is easier.

Save ponies: reduce the resource consumption of your django server - Raphaël Barrois

He works on a big django project. 300 models. Startup time is 12 seconds... Memory consumption 900MB. The first query afterwards after 4 seconds.

How dows it work? They use uwsgi. There is a simple optimization. By default, uwsgi has lazy = true. This loads apps in workers instead of in the master. He disabled it. It started up much slower, but the first query could then be handled much quicker.

Another improvement. They generate a fake (test) request and fire it at the server upon startup. So before the workers are started, everything is already primed and active. This helps a lot.

Djangocon: Graphql in python and django - Patrick Arminio

(One of my summaries of a talk at the 2018 European djangocon.)

For APIs, REST is the normal way. But REST is not perfect.

You can, for instance, have too many requests. If you request a user (/users/1) and the user has a list of friends, you have to grab the user page of all those friends also. You could make a special endpoint where you get the names of the friends, but can end up with many endpoints (/users-with-friends/1, /users-with-friends-and-images/1). Or with very big responses that contain everything you might need.

Graphql was created to solve some of these issues. You have a single /graphql endpoint, which you POST to. You post the data structure that you want to get back. There's the option of adding types. So you're not bound to pre-defined REST responses, but you can tell exactly how much or how few you need and in what form.

Almost every graphql instance has introspection enabled. You can discover the API that way, including which data types to expect.

In python, you can use the graphene library. From the same authors, there's graphene-django.

There is also integration for django REST framework in graphene-django. Quite useful when you already have all of your serializers.

For trying out a graphql API, https://github.com/graphql/graphiql is a handy in-browser IDE to "play" with it.

(He demoed it: looked nice and useful.)

What about security/authentication? Standard session based authentication. Or you can use an authentication header.

What about malicious queries? You could get big exploding responses by following a foreignkey relation back and forth (author->posts->authors->posts etc).

In the end, graphql is quite handy, especially when you're working with many developers. With REST, you'd have just finished one response when the UI people were already clamoring for other, different responses. That problem is gone with graphql.


Photo explanation: station signs on the way from Utrecht (NL) to Heidelberg (DE).

Djangocon: it's about time - Russell Keith-Magee

(One of my summaries of a talk at the 2018 European djangocon.)

Handling time and timezones is complex and painful.

It starts with leap years. Every four years, there is a leap year. Except every 100 years. Except except every 400 years. The latter two rules are from the gregorian calendar, which replaced the julian calander in 1582 (at least, in parts of the world...)

When did the "october revolution" happen? Well, either 25 oct or 7 november 1918. Russia still had the Julian calender at that time :-)

Year? Well, some countries use years based on the lunar cycle... Or they count from a different starting point.

In IT we had the Y2k problem. In 20 years time there'll be the 32 bit epoch overflow. It already crashed the AOL mail servers (!) years ago.

In python, there's the time module. It represents how your computer thinks about time. It isn't terribly useful if you actually want to do something with it that you're going to show to the user.

The datetime module is the one you'll probably want to use. But do you use a date or a datetime? A date doesn't have a timezone. The day Hawkings died, and you asked it on google, you'd get "he died tomorrow" if you asked it from the USA....

You'll need to use datetimes. With timezone info, not "naïve datetimes".

You absolutely have to use the pytz module. That is absolutely necessary. It is regularly updated. This year, it already had 5 updates. Countries change their timezone. Sometimes retrospectively.

He showed a couple of weird timezones examples. And he didn't even have to leave Australia.

Timezones are date sensitive. Daylight savings time, for instance. And as timezones can change....

Reading in dates: hard to get right. People write dates differently. 1/6/18 can be 1 jan 2018, 18 june 2001 etc....

And specifying the right timezone is hard. ISO8601 has a way of specifying the timezone as +08:00, for instance. But then you only know which of the 20 possible timezones it is because you only have the +8 hours, not the timezone...

Oh, there are also leap seconds. 23:59:61 sometimes is a valid time!

Oh, and don't communicate like "it will be released in fall 2018": on the southern hemisphere, fall is in the first half of the year.

A request:

  • Always include a year.
  • Always use the text version of the month (localized).
  • Always include a timezone.
  • Always use ISO8601 in logs.


  • A date means nothing without a time
  • A time means nothing without a date.
  • Both are nothing without an accurate timezone

If you thought everything was hard now, what when we start colonizing planets? A day on Venus is longer than a year on Venus :-)


Photo explanation: station signs on the way from Utrecht (NL) to Heidelberg (DE).

Djangocon: banking with django, how not to lose your customers' money - Anssi Kääriänen

(One of my summaries of a talk at the 2018 European djangocon.)

He works for a company (holvi.com) that offers business banking services to microentrepreneurs in a couple of countries. Payment accounts (online payments, prepaid business mastercard, etc). Business tools (invoices, online shop, bookkeeping, etc).

Technically it is nothing really special. Django, django rest framework, postgres, celery+redis, angular, native mobile apps. It runs on Amazon.

Django was a good choice. The ecosystem is big: for anything that you want to do, there seems to be a library. Important for them: django is very reliable.

Now: how not to lose your customers' money.

  • Option 1: reliable payments.
  • Option 2: take the losses (and thus reimburse your customer).

If you're just starting up, you might have a reliability of 99.9%. With, say, 100 payments per day and 2 messages per payment, that's 1 error case per day. You can handle that by hand just fine.

If you grow to 10.000 messages/day and 99.99% reliability, you have 5 cases per day. You now need one or two persons just for handling the error cases. That's not effective.

Their system is mostly build around messaging. How do you make messages reliable?

  • The original system records the message in the local database inside a single transaction.

    In messaging, it is terribly hard to debug problems if you're not sure whether a message was send or what the contents were. Storing a copy locally helps a lot.

  • On commit, the message is send.

  • If the initial send fails: retry.

  • The receiving side has to deduplicate, as it might get messages double.

You can also use an inbox/outbox model.

  • Abstract messages to Inbox and Outbox django models.
  • The outbox on the origin system stores the messages.
  • Send on_commit and with retry.
  • Receiving side stores the messages in the Inbox.
  • There's a unique constraint that makes sure only unique messages are in the Inbox.
  • There's a "reconcialation" task that regularly compares the Outbox and the Inbox, to see if they have the same contents.

For transport between inbox and outbox, they use kafka, which can send to multiple inboxes.

There are other reliability considerations:

  • Use testing and reviews.
  • If there's a failure: react quickly. This is very important from the customer's point of view.
  • Fix the original reason, the core reason. Ask and ask and ask. If you clicked on a wrong button, ask why you clicked on the wrong button. Is the UI illogical, for instance?
  • Constantly monitor and constantly run the reconciliation. This way, you get instant feedback if something is broken.

Photo explanation: station signs on the way from Utrecht (NL) to Heidelberg (DE).

Djangocon: don't look back in anger, the failure of 1858 - Lilly Ryan

(One of my summaries of a talk at the 2018 European djangocon.)

Full title of the talk: "don't look back in anger: Wildman Whitehouse and the great failure of 1858". Lilly is either hacking on something or working on something history-related.

"Life can only be understood backwards; but it must be lived forwards -- Søren Kierkegaard" So if you make mistakes, you must learn from it. And if something goes right, you can party.

The telegraph! Old technology. Often used with morse code. Much faster than via the post. Brittain and the USA had nation-wide telegraph systems by 1858. So when the first trans-atlantic cable was completed, it was a good reason for a huge celebration. Cannon, fireworks. They celebrated for three weeks. Until the cable went completely dead...

Normally you have a "soft lauch". You try out if everything really works for a while. But they didn't do this in case. They could only imagine succes...

Failures aren't necessarily bad. You can learn a lot from it. But at that time you didn't have agile retrospectives. An agile retrospective at that time was probably looking over your shoulder while you sprinted along the highway, chased by highway robbers...

Now on to Wildman Whitehouse. It looked like he had done a lot of inventions, but most of them were slight alterations on other peoples' work. So basically "fork something on github, change a single line and hey, you have a new repo".

But it all impressed Cyrus Field, a businessman that wanted to build a transatlantic cable. Undeterred by the fact that electricity was quite new and that it seemed technically impossible to build the thing. He hired Whitehouse to build it.

Another person, a bit more junior than Whitehouse, was also hired: William Thomson. He disagreed on most points with Whitehouse's design (and electrical theory). But there was no time to do any discussion, so the project was started to Whitehouse's design.

The initial attempts all failed: the cable broke. On the fourth try, it finally worked and the party started.

Now Thomson and Whitehouse started fighting over how to operate the line. Whitehouse was convinced that you had to use very high voltages. He actually did this after three weeks and fried the cable.

Time for reflection! But not for Whitehouse: everyone except him was to blame. But the public was terribly angry. There was an official "retrospective" with two continents looking on. Thomson presented the misgivings he shared beforehand. Engineers told the investigators that Whitehouse actually personally (illegally) raised the voltage. So it soon was clear that there was one person to blame... Whitehouse.

Whitehouse was indignant and published a pamflet... And nobody believed him and he stormed off to his room.

Some lessons for us to learn:

  • Open-mindedness is crucial. You must allow your ideas to be questioned.

  • Treat feedback sensitively. People get defensive if they are attacked in public.

    Before you do a retrospective, make sure it is clear that everyone did their best according to the knowledge in the room

  • Remember the prime directive.

  • There is no room for heroes in the team. It makes for a terrible team environment.

How did it end? They let William Thomson take over the management. The design and principles were changed. Also good: Thomson was nice to work with. Still, some things went wrong, but they were experienced enough to be able to repair it. After some test period, the cable went into operation.


Photo explanation: station signs on the way from Utrecht (NL) to Heidelberg (DE).

Djangocon: an intro to docker for djangonauts - Lacey Williams Henschel

(One of my summaries of a talk at the 2018 European djangocon.)



  • Nice: it separates dependencies.
  • It shares your OS (so less weight than a VM).
  • It puts all memmbers on the same page. Everything is defined to the last detail.

But: there is a pretty steep learning curve.

Docker is like the polyjuice potion from Harry Potter. You mix the potion, add a hair of the other person, and you suddenly look exactly like that other person.

  • The (docker) image is the person you want to turn into.
  • The (docker) container, that is you.
  • The Dockerfile, that is the hair. The DNA that tells exactly what you want it to look like. (She showed how the format looked).
  • docker build actually brews the potion. It builds the image according to the instructions in the Dockerfile.

Ok. Which images do I have? Image Revelio!: docker images. Same with continens revelio: docker container ls.

From that command, you can grap the ID of your running container. If you want to poke around in that running container, you can do docker exec -it THE_ID bash

Stop it? Stupefy! docker stop THE_ID. But that's just pause. If it is Avada kedavra! you want: docker kill THE_ID.

Very handy: docker-compose. It comes with docker on the mac, for other systems it is an extra download. You have one config file with which you can start multiple containers. It is like Hermione's magic bag. One small file and you can have it start lots of things.

It is especially handy when you want to talk to, for instance, a postgres database. With two lines in your docker-compose.yml, you have a running postgres in your project. Or an extra celery task server.

Starting up the whole project is easier than with just plain docker: docker-compose up! Running a command in one of the containers is also handier.

The examples are at https://github.com/williln/docker-hogwarts


Photo explanation: station signs on the way from Utrecht (NL) to Heidelberg (DE).

Djangocon keynote: the naïve programmer - Daniele Procida

(One of my summaries of a talk at the 2018 European djangocon.)

The naïve programmer is not "the bad programmer" or so. He is just not so sophisticated. Naïve programmers are everywhere. Almost all programmers wish they could be better. Whether you're

Programming is a craft/art/skill. That's our starting point. Art can be measured against human valuation. In the practical arts/crafts, you can be measured against the world (if your bridge collapses, for instance).

Is your craft something you do all the time, like landing a plane? Or are you, as programmer, more in the creative arts: you face the blank canvas all the time (an empty models.py).

In this talk, we won't rate along the single axis "worse - better". There are more axes. "Technique - inept", "creative - dull", "judgment - uncritical" and sophistication - naïve. It is the last one that we deal with.

What does it mean to be a sophisticated programmer? To be a real master of your craft? They are versatile and powerful. They draw connections. They work with concepts and ideas (sometimes coming from other fields) to think about and to explain the problems they have to solve.

The naïve programmer will write, perhaps, badly structured programs.

But... the programs exist. They do get build. What should we make of this?

He showed an example of some small-town USA photographer (Mike Disfarmer). He worked on his own with old tools. He had no contacts with other photographers. Just someone making photo portraits. Years after his death his photos were discovered: beautifully composed, beautifully lighted (though a single skylight...).

Software development is a profession. So we pay attention to tools and practices. Rightfully so. But not everyone is a professional developer.

Not everyone has to be a professional programmer. It is OK if someone learns django for the first time and builds something useful. Even if there are no unit tests. Likewise a researcher that writes a horrid little program that automates something for him. Are we allowed to judge that?

He talked a bit about mucisians. Most of them sophisticated and very good musicians. But some of them also used naïvity. Swapping instruments, for instance. Then you make more mistakes and you play more simply. Perhaps you discover new things that way. Perhaps you finally manage to get out of a rut you're in.

Some closing thoughts:

  • Would you rather be a naïve programmer with a vision or a sophisticated programmer without?
  • If you want to be a professional developer, you should try to become more sophisticated. That is part of the craft.
  • If you're naïve and you produce working code: there's nothing wrong with being proud of it.
  • As a sophisticated programmer: look at what the naïve programmer produces. Is there anything good in it? (He earlier showed bad work of a naïve French painter; his work was loved by Picasso.)

(Suggestion: watch the keynote on youtube, this is the kind of talk you should see instead of read).


Photo explanation: station signs on the way from Utrecht (NL) to Heidelberg (DE).

Djangocon: survival tricks and tools for remote developers - Alessio Bragadini

(One of my summaries of a talk at the 2018 European djangocon.)

He works in a company that has many remote workers. He is one of them. The core question for him: "how do I manage to work remotely in an effective way without much stress".

There is a difference between a remote-friendly company and a remote-first company. Remote-friendly is a company that still has an office. But you're allowed to work from home and you don't have strict work hours. Remote-first changes the entire structure/culture.

Can agile help?

  • Test driven development. First you make the tests. That's handy for remote teams. It sets strict boundaries where you can take over the work in a way that you do not have when sitting behind the same keyboard.
  • No code ownership. Anybody can work on everything all the time.
  • Shared "visual backlog" (boards and so).

But... "agile" also says that teams that work face-to-face are more efficient in conveying information. But note that the agile manifesto is many years old now.

Face-to-fase means proximity, but also truthfulness. So: no documents that can mean anything, but truthful conversation. Eh: we are now used to slack, skype, whatsapp. This is 99% of what face-to-face means. (You still miss body language, though, and the pleasure to be near to each other).

And, what is information? Discussion about the project, about code or design. Information about what moves forward: commits, tasks. Info about what moves backwards: bugs, regressions. All these things can be done online. Some of these can even be done better online.

The more you use these online communication channels, the more you become remote-first. Being in the office is almost accidental. The online channels become stronger if you have your machines post feedback there ("failed test!"). Perhaps even automate tasks that you can start via messages in your channel...

You need a shared repository that is accessible everywhere. A channel to communicate on. Automatic testing. CI. Etc.

Some comments:

  • There are some agile "ceremonies" like a daily standup and a sprint review. Do that, but online.
  • Explain what you're going to do and what you've done. Don't work in an invisible way.
  • Establish "work hours" even if you are not in a proper office. This is perhaps counter-intuitive to working remotely.
  • Important: keep the chat hannel open during work hours.
  • Do meet face-to-face from time to time.
  • Learn from companies that do remote-first: automattic, balsamiq.

Some tools they use:

  • Test driven development (unittests, selenium).
  • Infrastructure as code (VMs, docker).
  • In-house Gitlab as their git repository and project center.
  • Continuous integrations (with pipelines on gitlab). Due to the automated pipelines, no one has to do those tasks. Otherwise you often have a single person that has to do those kinds of tasks so that he feels a bit separated from the rest. Automate it away instead so that everybody can code.
  • Slack channel with integrations with gitlab and sentry. (via a Chatbot)
  • Gitlab boards, some trello boards.
  • Skype for "agile ceremonies" including the daily standup.
  • Google docs.

(He mentioned an article by Lee Bryant about slack, I guess this is it, but I'm not sure).


Photo explanation: station signs on the way from Utrecht (NL) to Heidelberg (DE).

May 24 2018

Djangocon: organizing conferences for learners, how we did it in Namibia - Jessica Upani

(One of my summaries of a talk at the 2018 european djangocon.)

For Jessica, it all began at PythonNamibia2015. She went there, not because she wanted to learn python, but because she was bored. And the conference was free. It had all changed by the end of the conference! Thanks to the organizers that inspired a lot of people there to become active with python.

In 2017, she helped organize a 'computer day'. Talks and panel discussions, poster presentations, software project presentations and workshops. It was aimed at kids!

Especially the panel discussions were aimed at the newcomers: trying to transfer experience. In 2018, there were separate workshop days, amongst other an introductory python course.

There are some differences from organizing a conference for adults:

  • You need to write letters to parents! Convincing them to send their kids to the conference.
  • Behaviour. You need to follow the behaviour of the kids. They're less well-behaved than a room of adults. Adults sit still, kids move around and look for attention. The one giving the presentation needs to be a good teacher, otherwise they won't be able to keep the kids' attention.
  • Different expectations. Kids expect fun and games, they don't expect talks. So you have to tell them beforehand what is going to happen. And you have to adjust the program.
  • Fun and games. Yes, you need it.
  • Speakers. You have to help the speakers. You have to check their talks beforehand: is the content easy enough for the kids? You don't want their eyes to glaze over and their attention to wander.
  • Funding. Important! It is also very hard. She hasn't found any local sponsors till now.
  • Decorations. Yes, put up balloons and other decorations!

What's important:

  • Connect to knowledge they learned in their classes.
  • But: give them more than they get in their regular class.
  • Inspire computing career choices.
  • Passion and knowledge.
  • Get them to challenge themselves. A great way is to let them build something that others have to try out: then it has to be pretty good!

Ok... all this is quite some work. Why go through all that trouble?

She teaches at a school with 215 students. But there is not a single computer. How to let those students get into contact with computers? Organizing such a computer day at the university helped. They could use the university's computers in the weekend that way. And it helped get some sponsorship for computers for her school.

Help is needed here!

Thanks for the python and django foundations, as they were the two that made the computer day 2017 possible. In 2018, they were more sponsors: thanks!


Photo explanation: constructing a viaduct module (which spans a 2m staircase) for my model railway on my attic.

Djangocon: slow food digest better (maintain an old project) - Christopher Grebs

(One of my summaries of a talk at the 2018 european djangocon.)

Full title: "slow food digest better - or how to maintain an 8.5 year old python project without getting lost". Christopher had to maintain such a project - and actually liked it. It was https://addons.mozilla.org, actually.

It started out as a quickly-hacked-together php project. Now it an almost modern django project. The transition from PHP to the django version took almost 16 months. During that time there were bugs, translation errors, downtime: irritating. The site went fully live in january 2010.

The big advantage of the move to django was that lots of tests were added at that time. The site wasn't anything special. Mostly django. Still quite some raw SQL from the old system. Celery for some tasks.

Mozilla at one time had the "Firefox OS" for mobile phones. For that, they build the "firefox marketplace". The work was based on the addons.mozilla.org code, but with some weird hacks based on which site it was running... During that time the addons.mozilla.org website itself was pretty much left alone.

In 2015 they had to re-animate the old codebase. At that time it was running django 1.6, barely, as lots of deprecated features were still used as long as they stayed available. The javascript code had almost no unit tests (and the site at the time was javascript-heavy).

So: complete rewrite or incremental improvement? They chose incremental improvement. Rewriting a huge site from scratch for a small team... no. And with the existing system they at least had the advantage of all the existing unittests!

The balance they had to make was between "removing technical dept" and "new features".

What they did was create a new react-based frontend as a single page app. This got released in december 2017. So they incrementally rewrote the backend (where they had unittests) and did a full rewrite of the frontend (which had no tests).

One thing they used a lot: feature flags/switches. They used "waffle" for that. It makes it much easier to revert broken implementations as you only have to flip a swich back.

Beware of third party dependencies. They can be a great pain. Especially when you want to upgrade your django version. On the frontend you have a similar problem: there you can be inundated by javascript versions and updates and problems. Make sure your dependencies are up-to-date and maintained. If not, fix it or move to other libraries.

They steered their django upgrades with waffle feature flags. Once the new django version was fully in production, they could remove the feature flags for the old version.

A quality assurance safes lives. Unittests are good, but a real QA team that really tests it discovers lots of problems. And purely the fact that you need to explain the upgrade process to the QA engineers already helps.

And... don't panic. You're there for the long run. Great food needs time, why should your software be different?


Photo explanation: constructing a viaduct module (which spans a 2m staircase) for my model railway on my attic.

Djangocon: strategies to edit production data - Julie Qiu

(One of my summaries of a talk at the 2018 european djangocon.)

She works at the catalog team of "spring", a clothing website.

Internal tools are often not available. There are always edge cases. Time-sensitive changes are sometimes needed ("right now").

You could just do a quick SQL query in the database. Normally, a colleague will look over your shoulder and double-check what you're about to type. But when it is friday afternoon and your highest-paying client wants a last-minute change.... They do have a collections of horror stories...

Here are some strategies:

  • Develop a review process for manual edits. What they've done is to create a spreadsheet. You'd have to write in there your name, the sql code, what you want it to do, who you want to review it. Only after the review, you are allowed to run it on the server.

    The advantage is that it is easy to implement and that you get an audit trail. You also teach engineers what is the right thing to do.

    A disadvantage is that you still can get mistakes. And it is fine for smaller changes, but not really for elaborate SQL and long-running queries.

  • Write scripts and run them locally. Write a python script to make the change. Add commandline arguments so that you can re-use the script. Then you have to connect it to the database and run it.

    Advantage: it is also fine for more complex changes.

    Disadvantage: you run it locally, so logs are only available locally. You can still make mistakes. The local scripts are local: there's no review for them. And you can have connection issues.

  • You can run the scripts also on an existing server. This way, you generally don't have the connection issues. You do have to run it in 'screen'.

    After writing the script, you have to get the script onto a server. SSH there and run inside a session. Julie normally runs it on the jenkins machine. But.... one of her scripts once ate up all CPU resources, so jenkins was down...

    Advantage, in general: you can have long-running scripts. And you have a much more reliable network connection.

    Disadvantage: you can affect the resources on the server. And you have to copy your script to the server.

  • Use a task runner. You can use jenkins to run scripts.

    Now you have to get your script reviewed like the rest of your code. The latest version is automatically on jenkins. Jenkins provides a way to pass arguments to such a script.

    A big advantage: the output of the run is stored in jenkins. You have an audit trail. And you have code review.

    Disadvantage: it is hard to manage credentials. Also: you apparently can connect to your production database from your jenkins test environment. This is asking for accidents to happen.

  • Then she decided to write a (jenkins) "script runner" service. So it was customizable.

    Again: write the script and get code review and run tests. Then you can run it with a nice user interface in jenkins. The custom script runner could be pre-configured with the various configs (dev, staging, production), so that managing the credentials was easy.


Photo explanation: constructing a viaduct module (which spans a 2m staircase) for my model railway on my attic.

Djangocon: automated spell checking in django projects - Jakob Schnell

(One of my summaries of a talk at the 2018 European djangocon.)

We are humans and we make typos. So there are typos in our code.

The two common places for typos are documentation and the user interface.

Documentation is normally only provided in a single language and it are large text files, so spell checking is relatively easy. Django documentation is often build with Sphinx. For that, there is a sphinx extension: sphinxcontrib-spelling. You can even integrate it in your CI as a post-build check.

There will be words that are correct for your project but that aren't in the regular dictionary: for that there's a local "wordlist" you can use.

For code (and our GUI), it gets more complicated. You would have to read python code, css code, html code, javascript code... hard.

But there is a solution! Translations. Once your project gets bigger, you probably want to start translating it. Once you have set up your translation mechanism (gettext), you have all your strings gathered into one place: the .po files. Hurray, now we can do spell checking.

Gettext is the standard mechanism in Django to deal with translations. You'll see lines like from ... import ugettext as _ in your code.

He wrote a tool for it: potypo. polib + pyenchant = potypo. Polib can read and write the "gettext" *.po files. pyenchant is an interface to libenchant.

You're probably familiar with ispell or aspell (an ispell that fits better to unicode). myspell is openenoffice's spellchecker, hunspell is a variant on this. For English, "aspell" is probably best, for other languages "hunspell". Libenchant is a library that wraps them all. And pyenchant provides a python apy to libenchant.

When you start using it, you'll have to install language packages like myspell-de-de and aspell-en. Then add a bit of configuration and potypo can start check your spelling. If desired, it can fail your build in CI. You can also switch that off for specific languages (for instance if you've just started translating).

Wordlists? You have multiple languages, so wordlists can be in a directory. wordlists/en.txt, wordlists/de.txt. You can also put just a wordlist.txt inside the translations' locale/de/ directory.

The "pytypo" project is quite new, but it is already used in several projects. Ideas, features, pull requests: everything is welcome!


Photo explanation: constructing a viaduct module (which spans a 2m staircase) for my model railway on my attic.

Djangocon: protecting personal data with django (because it's the law) - Will Hardy

(One of my summaries of a talk at the 2018 european djangocon.)

Will is a software developer with a law degree. Now that we have the GDPR, his law degree is suddenly very relevant. GDPR takes effect on 25 May 2018.

What is the GDPR? It is a law that regulates the use of personal data.

You'll probably have had lots of emails from companies telling you that they'll be good with your data and asking whether they're still allowed to use it.

He encourages you to read the actual regulation. The first part is quite readable. The actual articles are quite detailed, but only the first 34 are relevant for us. He thinks we have a professional duty to be on top of this. We have to know about it.

As programmers, we're in the front line. We might be the ones that can best advise the company on how to comply. We ought to know the details. If you help your company, you're valuable to your company, so...

He has three categories in his talk: terms, rights, tasks.


  • Terms in the legal world aren't defined as rigorously as in software standards. "Personal data is any information relating to an identied or identifiable natural person". Ok.... is an IP adress personal data, yes or no?
  • "Processing means...", right, basically, it is everything you do.
  • Data controller, that's what you are when you know something in a professional context about someone else. Info between friends is OK, for instance.
  • Processor: someone who does something with personal data belonging to a "data controller". Freelancers: this is for you.
  • Special categories: really watch out when you store religion, sexual orientation or so.
  • Profiling: also watch out. You might be getting too close to "special categories".


  • Transparency. You now have the right to know what they know about you.
  • Access. You get to see what they now about you. As a data controller, you also have to make clear what you do with the data.
  • Rectification. Does the system allow itself to be changed or updated? If you have a django website with an admin, you're fine.
  • Erasure. Deleting a user, can you do that? Does that include your backups?
  • Data portability. In a structured, commonly machine readable format. Django can help
  • Restriction of processing. No deletion, but more "put me on hold".
  • No automated decision-making. You have the right to be approved/disapproved by a human being
  • Right of consent.

Tasks for us

  • By design and default. Learn to do it properly. If you work with django, follow recommended django practices and feel that you're behaving yourself, you're probably OK.

    Important here is "data minimalization". Don't pass along full user objects to other systems. Even not the userid. Generate a UUID or so.

    Separate personal data completely. "Pseudo-anonymization".

    For a medical database, does your database support staff need to see a person's name? No. Only the doctor needs to know that. Then you might be better off encrypting the name.

  • Erasure. Can you split the backups? A separate one for personal data and one for the rest? That might make zapping personal data easier.

  • No discrimination. You cannot discriminate with prices on areas where people live, anymore. If you have algorithms that make decisions, watch out for biases.

    Note: gender and age are not included here! So special prices for older or younger people are fine. But, again, watch out for indirect discrimination. There are other laws that you have to take into account.

    (See my summary of the great talk on biases)

    Your algorithms will get better because of it.

  • Explain machine learning. If you make an automatic decision, you might have to explain it. If it is an unclear pile of a neural net, it might be hard to explain...

  • Anonymization. True anonymization is rare. And hard. The answer you have to ask is "is reidentification reasonably likely". And as a programmer, you're probably the only person that can answer it.

    Again, anonomyzation is hard. You'll probably have to get outside expert help.

  • Breach notification. If there is a breach, you have to report it. Otherwise you are liable. Even putting too many people in an email's CC field could be a breach...

What could django do?

  • Per-user encryption. So that you can delete a per-user key so that the encrypted personal data isn't readable anymore.
  • Documentation.
  • Tag personal data.

The current situation isn't clear yet. In a few years it probably will be.


Photo explanation: constructing a viaduct module (which spans a 2m staircase) for my model railway on my attic.

Djangocon: growing old gracefully, on being a career programmer - Carlton Gibson

(One of my summaries of a talk at the 2018 european djangocon.)

He's a longtime django user. "The web framework for perfectionists": yes, that's him.

He has also build lots of backends for IOS apps. For that, he used django REST framework. He got involved on the mailinglist and on stackoverflow. Now he's a core team member of django rest framework. He also started maintaining django-crispyforms and so. And now he's the "django fellow".

So far, so good. He has a job. Nothing million-euro-making, but fine. The problem: he's getting older. There was a post on hacker news: "before you're 40, make sure you have a plan B".

You might get into problem when searching for django jobs: "you're older, so you're more expensive, so we'll take a younger person". You also might have a family, so moving (especially working abroad) is harder.

There's a common path: become a manager. But he won't do that. He's a good programmer, but not a good manager. He wants to stay productive and creative.

So what if you just want to keep programming? He has some generic strategies:

  • Look outside the tech bubble. The software tech bubble. "Software is eating the world", so who's going to program that? Look at traditional industries!

    (Note by Reinout: I'm working in a civil engineering construction firm now!)

  • Always ask "what's next"? If you're a freelancer, you can handle an occasional bad project if you have a good pipeline. But if you have a good project and a bad pipeline, you have a problem.

    Also make sure you develop yourself constantly. Become more valuable. Become more valuable to your employer. Make sure he knows how valuable you are (especially how valuable you could be to other companies).

There are also specific strategies:

  • Be diligent. The number one priority is self care. Care about the pace. Don't do "death march" projects. You cannot do 100 hour workweeks until your pension! Keep a good pace.

    Similarly, make time for your family if you have one.

  • Be diligent: eliminate distractions. If you won't work yourself to death in 100 hour workweeks, you have to make sure you work hard in the time you do work. Look hard at your distractions. How often do you check your email? Facebook? Your phone?

    Procrastination is a big problem. You might have a big pile of underspecified work. The real job would be to specify/clarify the work, but it is much easier to let yourself be distracted.

    Some apps he uses to prevent distraction:

    Combined, these three tools help him eliminate

    Regarding his phone? His tip is delete everything. Don't have all those apps in there that can distract you. News sites you check 20x a day: what is the use to have that on your phone?

  • Be dilligent: develop good habits. By eliminating distractions, you've made time for other activities. What do you want to do?

  • Be prolific. Do one thing. And then another. And then another. The individual things you do might not be big of itself, but it all adds up after 15 years.

    He himself is not a superstar developer, but he's done a lot!

    A tip: contribute to open source. There's a low barrier to entry (in theory at least). And it is visible! It is stuff you can talk about. Anything you stick on github, you can point to it. If your work-time code is all private, you cannot show it. At job interviews, you might be asked for a code sample. If you've got lots of stuff on github, you won't get that question.

    Contributing to open source is a "slow burner". It is not something you can do quickly in order to make a good impression at a job interview. It is something you have to do for quite some period of time. Small contributions over time in the end up as a sizable and visible contribution.

Some comments on open source:

  • One thing to watch out for, when contributing to open source: make sure you don't burn out. People will ask the same support questions over and over and over again. There are risks of "contributing to open source is eating my time" or "is making me sick" or "limiting the time with my family". You don't want that.

    That's the basic purpose of the django fellow: it is a paid job to do the menial work so that it doesn't burn out the other developers.

    As a regular contributor, limit your time beforehand.

  • Get your employer to help. Open source is often a better alternative than building your own.

    When working on open source code, you'll also learn a lot. And it will improve you. And it will increase your value for your company.

    They should be funding django, django rest framework, etc. It is much, much cheaper to fund those projects to get your bugs fixed than to hire programmers directly to fix it.

  • Specifically for django: when you contribute a pull request, you'll get lots of comments. They're necessary to keep up django's quality. But it can be a bit discouraging. But don't dispair! And let Carlton help you work through the comments. That's his job as django fellow.

To close it off, the secret weapon: be social. Get involved in open source. Talk to people. Go to meetings. Genuine interaction instead of websites-with-an-algorithm.

It will help you find a good employer. As an employer, it will help you find good programmers. There are many good companies that are good for their employees. You'll find them by talking to people! There are many good programmers looking for a nicer job, you'll find them by talking to people.


Photo explanation: constructing a viaduct module (which spans a 2m staircase) for my model railway on my attic.

Djangocon: taking channels async - Andrew Godwin

(One of my summaries of a talk at the 2018 european djangocon.)

Channels, started in 2015 as "django-onair", had its 1.0 release in 2017. It used twisted, ran on python 2.7, and django runs synchronously.

Python 2.7 undermined it. Python 3 has asyncio support, 2.7 has not. Because of that, channels had to be too complex. The design was wrong.

Now, there's channels 2.0. It requires python 3.5+. Native asyncio! Much simpler to deploy, also. It was quite a big rewrite: 75% was changed.

A big challenge was that django had to become partially asynchronous. The regular django ORM, views, middleware, url routing is still synchronous. Parallel to that, there's channels (ASGI) middleware and so. Two separate worlds.

But still, there are a few contact points. Towards the ORM, for instance. So he needed two functions, sync_to_async and async_to_sync, to move between the two worlds. They took two months to write! Synchronous code has to run in threads. The ThreadPoolExecutor does most of the hard work.

Both async and sync code are useful. Channels lets you write your code as both. Async is hard, so you don't want to have to be forced to use it. Channels' two functions make it possible.

But: the async interface is separate from the sync interface. You just cannot provide both through one API.

He has a blog post that further explains his thoughts about handling async and sync code: https://www.aeracode.org/2018/02/19/python-async-simplified/

ASGI. WSGI, web service gateway interface, is used by all python web frameworks for handling requests/responses. There's one problem: it is synchronous. And you cannot have something that starts synchronous, is async in between, and ends up call the synchronous ORM: it will block the whole thread/process. You have to start asynchronous.

So: WSGI, but then async. So: ASGI :-) It is intended for general use, just like WSGI. The core is an Application object with an async __call__(self, receive, send) method.

It is "turtles all the way down": routing is an ASGI app. Middleware is an ASGI app.

What does this mean for django? Should it be in core? The main question is "how much can we make django async". You could progressively change pieces of django to be async and call it from synchronous code. But....

The problem, he thought, was the ORM. What would an async ORM look like? Is it even a sensible endeavour? Note that it has to be done in small steps. But after recent talks, he thinks it could be done.

Another question: do we really need to replace WSGI? How much demand is there for long-polling and websockets? A new standard (ASGI) is another new standard. And an extra standard is not necessarily good.

Websockets are a niche. Long-polling is less of a niche, but still a niche.

For ASGI to become a standard, you need multiple servers that implement it (apart from "daphne", there is now also "uvicorn"). And you need more frameworks that use it.

Another question: do we want to have everyone writing async? It is a pain to debug and hard to design. What is the balance? He would like for django to keep the existing sync interface for the majority of the cases. And if you need more power, that you then can dive deeper. Just like in many other cases.

So: if you have an opinion on where django should be going, talk to him and to other developers!


Photo explanation: constructing a viaduct module (which spans a 2m staircase) for my model railway on my attic.

Djangocon: ORM, the sequel - Katie McLaughlin

(One of my summaries of a talk at the 2018 european djangocon.)

SQL. The "language to talk to databases in a structured way".

The ORM. Object Relational Mapper. The magic that makes it all work in django (even though there's no magic in there).

The talk is about the experience of experienced programmers that, for the first time, have to dive into a django project. She used http://glasnt-orm.us.aldryn.io/ ("unicodex") as an example.

So. There's a missing icon on the sample page. You have to debug/fix that as an experienced-programmer-without-django-experience. You're used to SQL, but not to the django ORM.

You get a tip "use the shell". Which shell? "The c shell? bash?". No, they mean manage.py shell. With a bit of _meta and some copy/paste you can get a list of the available models.

You could do the same with manage.py dbshell, which would dump you in the sql shell. List the databases and you get the same answer. Pick a promising table and do a select * from unicodex_codepoint.

You can do the same in the django shell with:

from unicodex.models import Codepoint

The rest of the presentation was a nice combination of showing what happens in SQL and what happens when you use the ORM.

Once you get to the double underscores, for following relations and field lookups, the ORM starts to get more useful and easier than raw SQL:


It was a fun presentation. I can't really do it justice in a textual summary: you should go and look at the video. It is much more convincing that way.

There's a whole part about Q() object and the magic ways in which you can combine it with ~, & and |.

How would it translate to SQL? You can show the latest SQL query:

from django.db import connection

At the end of the presentation, she went back to the original usecase and started bughunting with the ORM. In the end it was a bug (a unicode 'bug' character) at the end of a filename :-)


Photo explanation: constructing a viaduct module (which spans a 2m staircase) for my model railway on my attic.

Djangocon keynote: 23 years without a 'proper job' - Rachel Willmer

(One of my summaries of a talk at the 2018 european djangocon.)

Rachel has used django since it was created, but this is her very first djangocon.

She hasn't had a "normal" salaried job for 23 years. She's been freelancing, self employed, some contracting to the Scottish government, had her own company, etc. So here are some tips for those that think about such a life.

We as programmers are very lucky. We're very flexible. We can work anywhere we like at anytime we like. As a bus driver, you have to show up when the bus timetable says.... We have lots of freedom, we 'only' have to arrange it so that who we work for agrees to it.

She has a side project, https://luzme.com, for searching for low ebook prices. Side project? Not really. She uses it to try out new techniques. Firebase, django channels, etc. Fun! And it helps her learn a lot more than when she would have just followed tutorials.

Ok, back to you. You want to do something for yourself. Perhaps your own company? If that is what you want, you have to learn how to run a company. You have to learn how to do finances (otherwise you work for your accountant instead of the other way around). You have to learn how to get customers. You have to learn to...

Tip: start small. Build a wordpress plugin or write an ebook. One-off. No support (though you could let people pay for it).

Support: that's service. Real support. Or "software as a service". The good thing: recurring revenue. But: it costs time. Your time. And that is your most precious resource. So don't treat your time as being free. Charge for it (also internally in your company), as if it

Publishing. Blog, vlog, podcasts.

How can you monitize?

  • Sales of books (small percentage on books bought via amazon via your site).
  • Affiliate commission.
  • Advertising. Especially if you're working in a focused market.
  • Sponsorship. You see it a lot in podcasts.
  • Data analytics. She isn't a dirty facebook, but though selling ebooks, she has a lot of data on buying books. She's able to sell that for serious money.

Some examples.

  • The documentation for "meteor" (real-time javascript web apps) wasn't good. So two people wrote a good book on it. They made 300.000 in 1.5 years and self-published it.
  • Anyone used 'balsamic'? Nice tool for wireframing. (I used it myself too). He started out as a solo founder. 2 million in profit two years ago.
  • You don't have to be from the USA to get it to work. 'patio11' did it from Japan.
  • "I cannot do this". Yes you can. Look at Jon Morrow. He cannot move from the neck down and still makes lots of money writing and has lots of fun.

Yes, you can:

  • OK to start small.

  • You don't learn until you start to fail.

  • Don't be afraid to think big.

  • Choose business-to-business and charge more.

    Regular customers want everything for free or cheap. Businesses are used to spending money to get something of value.

  • Choose the customers you want. Define the customers you want.

    Don't be afraid to say "no, you're better served by someone else" to a customer you don't want. If you say "yes" to "anyone" at "any price", it won't work.

  • Charge more. Yes, really. And even more. If you do valuable work, it is valuable.

    You have to get past your "imposter syndrome".

  • Above all: start a mailing list.

There are four things you need to learn to say: I don't know, I need help, I was wrong, I'm sorry. It is OK to say those things. (She has it from a book "still life", by Louise Penny).

She's a veteran programmer, but she doesn't know everything. So it is fine to look something up. It is OK to say you're sorry (because she deleted a whole project once: she apologized to her boss and therefore she was relieved enough to be able to think about the backup she could restore... :-) )

Also important: yet. I don't know yet. I can't write ruby. Yet. Next week, when I have the project, I'll be able to.


Photo explanation: constructing a viaduct module (which spans a 2m staircase) for my model railway on my attic.

Older posts are this way If this message doesn't go away, click anywhere on the page to continue loading posts.
Could not load more posts
Maybe Soup is currently being updated? I'll try again automatically in a few seconds...
Just a second, loading more posts...
You've reached the end.

Don't be the product, buy the product!