Why You Should Pin Your Dependencies by My Mistakes

Have you ever been bitten by not pinning your dependencies in your django project? If not be glad, and come learn from my problems.

Pinning your dependencies is important to solve future unknown issues, better the devil you know and all that.

In this weeks video I talk about 3 times I had issues. They are either not pinning my dependencies, a weird edge case with pinning and python, and not really understanding what I was doing with pinned dependencies.

Why You Should Pin Your Dependencies


Different types of testing in Django

Testing is always one of those topics that can be interesting to talk about. There are a lot of different opinions on testing so it can be fun.

Django comes with some great tools for testing, so this week we will talk a little bit about the different types of tests. Then expand on that with how that relates to Django.

I also present to you a new type of test at the end of the video that I have been using, special thanks to a co-worker for coming up with the idea. It is really specific to django, and I haven't heard of others doing it.

Different Types of Tests with Django


5 Reasons to Use Class Based Views

Anytime anyone brings up Class Based Views, generic and otherwise, it is similar to the Vim vs Emacs debate. So lets pile on a bit more.

But not really. In actuality, I feel like there are good, solid, and legitimate reasons to use Class Based Views which get passed over in the middle of arguments.

In this weeks topic I talk about 5 of those reasons I think people should use Class Based Views. In reality there are more, but I wanted to keep the video short'ish.

5 Reasons to Use Class Based Views


How to Vett Django Apps

There are a lot of 3rd party django apps that people put out which makes our lives easier, but are they good?

There are a lot of ways to evaluate them, mostly it seems a lot of people use intuition.

In this weeks video I go over several ways of how I go about vetting projects before use in my projects.

How to Vett Django Apps


How I Deploy Django Day-to-Day

There are a lot of ways to deploy Django so I think it is one of those topics people are really curious about how other people do it. Generally, in all deploys you need to get the latest code, run migrations, collect your static files and restart web server processes. How yo do those steps, that is the interesting part.

In todays video I go over How I deploy Django day to day, followed by some other ways I have done it. This is definitely a topic you can make as easy or complicated as you want.

Here is the link again: https://www.youtube.com/watch?v=43lIXCPMw_8?vq=hd720


5 Reasons I don't think you should use the django admin

Have you ever needed to quickly modify data or look it up. But while the task is simple the entire process of changing one value frustrates you?

Well, this is very common to me, bad UX is annoying. I quite often have to either lookup, or edit, two disparate, yet related pieces of data, and sometimes it is an exercise in frustration.

An all to common occurrence is I just needed to check the edit history of a "Company" our database. This is a simple process, go to the admin find the Company model, do a search and you have your answer.

Except, it isn't that easy in reality. Lets run through what really happened.

I went to "http://superawesomesite.com/admin/" and logged in.

I then looked at how my model options extended well below the fold of my browser so I had to scroll. No problem I'll just hit "ctrl+f" and search for it. WTF!!! where is my company model?

I then proceed to scroll down and finally find it only to remember it was pluralized. If I had searched for "Companies" I would have been good to go, grrrrr.

I click into and see the list of companies available to me. I see that there is a search feature at the top so I do a search for the relevant information. Unfortunately, I don't know the company name that is what I am trying to find out.

Except, we aren't filtering our search in the admin based on that field. So no results show up.

No problem I can do a list filter on the side. So I set the correct filters and still no luck because someone didn't set all the metadata that was supposed to be set for the company. Again grrrrr.

Finally, I abandoned the admin and opened up the django/python shell and did a query with the model, and in about 30 seconds had the record I needed. Took the id and plugged that into the django admin and I was good to go.

This was an exercise in frustration because the 30 seconds that was taken during development to decide which fields should be filtered didn't let that person imagine the field I needed. Also the fact that whomever added the company didn't add all the correct information, made it almost impossible to find the data. In the end I had to go outside this system that is hailed as one of the greatest tools for django to get the information I needed.

I propose we as a community reduce our reliance on the django admin, especially in production.

I also created a Video on the 5 reasons we shouldn't use the django admin.


Django Model Inheritance - Multi-Table Inheritance

In the last post we talked about Model Inheritance with regards to Abstract Model Classes, it was a great introduction into using OOP with your models to keep your code slim and tidy. You can read it at: Django Abstract Base Class - Model Inheritance

In this post we are going to discuss Multi-Table Inheritance. We will create a base model, that is not an abstract model, and inherit from the model. Then we are going to show you how to get and reference data from that models inheritance.

Multi Table Inheritance

When we talk about multi-table inheritance we are referring to having a top level table with all the fields that are core to that context. Then we are going to have other tables with the more specific fields. You select from the table you want, and then do a join on a relationship to get the parent table data. Since Django does this through models and code, we will talk about it in those terms.

In our example we are going to have a Ticket Model.

class Ticket(models.Model):
    title = models.CharField(max_length=255, blank=True)
    description = models.TextField(blank=True)
    name = models.CharField(max_length=255, blank=True)

This model is a good piece of code because no matter what type of ticket we ever have we are going to have a title, description, and name. This is normal for any OOP we would do as a base class. Since we have this base model we can subclass it to have more specific models with data associated with them.

class InternalTicket(Ticket):
    user = models.ForeignKey('auth.User', null=True)

class CustomerTicket(Ticket):
    product = models.TextField(blank=True)

Now we have 3 models available to us: Ticket, InternalTicket, and CustomerTicket. Each one has a title, description and name as fields. However, CustomerTicket does not have a user field and InternalTicket does not have product field. Also both InternalTicket and CustomerTicket are stored in separate database table from title, description and name. So if you looked up the tables in the database you would see a FK reference to the ticket table, id field and the field you declared. Everything else lives in the ticket table.

Getting and Creating Data

With all that said lets look at creating some data and getting it back out. Because that is really when it makes sense on how it works.

In [1]: CustomerTicket.objects.create(product='Videos', title='Odd Playback', description='doesnt play back fast enough', name='Jane Smith')
Out[1]: <CustomerTicket: CustomerTicket object>

We have created a CustomerTicket object with all of our fields set, and it succeeds. (Further proof it works)

In [2]: Ticket.objects.all()
Out[2]: [<Ticket: Ticket object>]

Here we are getting all tickets from the ticket model, which is including our CustomerTicket we create above.

In [3]: ticket = Ticket.objects.get()

In [4]: ticket.customerticket
Out[4]: <CustomerTicket: CustomerTicket object>

If we reference the customerticket property, which is the related name, it will return back the model so we can get the data from the model. We can look at that in action.

In [5]: ticket.customerticket.product
Out[5]: 'Videos'

In [6]: ticket.description
Out[6]: 'doesnt play back fast enough'

Finally, you can see the description is still on the ticket level and not the cutomerticket model.

How does it work

From the django docs:

multi-table inheritance uses an implicit OneToOneField to link the child and the parent

This means there is a one-to-one relationship between the ticket record and the corresponding subclassed models. Django just handles the translation of it from multiple tables to a single model so you don't really notice it.

CustomerTicket object

As our final look we can look at how the CustomerTicket object we created earlier references the ticket object.

n [1]: customer = CustomerTicket.objects.get()

In [2]: customer.description
Out[2]: 'doesnt play back fast enough'

As you can see django just handles the fact that description is on the model, even though the description data is in the ticket table.

In [3]: customer.ticket_ptr
Out[3]: <Ticket: Ticket object>

The .ticket_ptr property is the reference to the ticket object so if you need to actually use that model you can.

In [4]: customer.ticket_ptr_id
Out[4]: 1

ticket_ptr_id holds the id of the Ticket object in the database giving you another avenue for getting information out of the appropriate place.

Conslusion

You don't often need to use Multi-table inheritance, but when you do need to use it, it is nice that django makes it super simple to use. I have only seen a few instances where it makes sense, tickets being one, and social media posts as another. It provides great value in giving you core data, then when needed get the rest of the data. It does this while allowing you to use the models as separate models as if the inheritance didn't exist. Finally, another way to think of it is as data being used with polymorphism.


Django Abstract Base Class - Model Inheritance

Model inheritance is a very useful and powerful feature of django, but used incorrectly it can cause a lot of confusion. Lets go ahead and take a look at some of those.

Have you ever been looking through other peoples code and seen a model class Meta with abstract = True? That is the first type of inheritance we are going to look at, abstract base models. There is also multi-table inheritance, proxy models, and multiple inheritance available.

Abstract Base Classes

In this post we are going to focus on Abstract base classes. This is the easiest type of model inheritance to really understand because it works very much like normal inheritance. In OOP when you inherit from another object you get all of its members plus your own. You can modify those members or leave them the same. Abstract base classes are the same.

Here are two example classes

class Customer(models.Model):
    name = models.CharField(max_length=100)
    email = models.EmailField(max_length=100)
    address = models.CharField(max_length=100)
    purchase_history = models.ForeignKey('cart.Invoice')

class Staff(models.Model):
    name = models.CharField(max_length=100)
    email = models.EmailField(max_length=100)
    address = models.CharField(max_length=100)
    bio = models.TextField()
    position = models.CharField(max_length=20)

As you can see these two models are legitimate models with fields, but they do have common elements. In a lot of cases it is fine to leave them this way. However, if we use these same fields yet again it would be a good idea to use an abstract base class.

class ConactInfo(models.Model):
    name = models.CharField(max_length=100)
    email = models.EmailField(max_length=100)
    address = models.CharField(max_length=100)

    class Meta:
        abstract = True

class Customer(ContactInfo):
    purchase_history = models.ForeignKey('cart.Invoice')

class Staff(ContactInfo):
    bio = models.TextField()
    position = models.CharField(max_length=20)

We now have a ConactInfo abstract base class because we added abstract = True to its Meta. This will tell django and the migration system this isn't a model we can use to store data with.

However, we can inherit from it so that each model that we subclass with it has the fields, methods, and properties of the abstract model. At this point if you did ./manage.py makemigrations it would create a migration file creating only Customer and Staff tables in the database.

Will it Work with Existing Models?

Lets say you have several existing models in your code base you want to abstract into base classes. Will it "just work"? Yes it will. Once I found out about abstract classes I went and made several then ran a makemigrations and nothing happened because to django nothing changed you just refactored you code.

Conclusion

Hopefully this gives you a place to start with doing model inheritance. It is very useful, but much like OOP don't go crazy with it and have a lot of abstract classes. You can be so abstract you have no idea what is going on.

My rule of thumb, whether good or bad, is if I have more than 3 fields repeated in only 2 models I just leave them alone. If I need to add them to a third model I will evaluate if it makes sense to use an abstract model. If I can name the abstract model and it makes sense like ContactInfo then there is enough context to create the new model, so I will. Otherwise I will put it off until the code is screaming for an abstract base class.

Stay tuned for the next post on Multi-table inheritance.

What has your experience been?


GoDjango Podcast Episode 1 - Interview with Luke Crouch

Listen in as we talk to Luke Crouch about upgrading from Django 1.4 to 1.7, MDN and discuss a potential move from a dedicated data center to AWS.


Working with Environment Variables in Python

If you look at modern code deployment practices, like in the 12 factor app, environment variables are very important for keeping secret information secret, or server specific information set for that server. All while not having long crazy settings in a settings file for every single server and computer. It helps keep things slimmer in your code along with the following quoted from the 12factorapp site:

    * Resource handles to the database, Memcached, and other backing services
    * Credentials to external services such as Amazon S3 or Twitter
    * Per-deploy values such as the canonical hostname for the deploy

What Are Environment Variables?

Environment variables are a key value pairs that can affect how a program runs. They need to be set at some point before a process is run so the process can read them in and act accordingly. A lot of times in production environments your database name and password are set as environment variables so that information does not end up in a code repository somewhere.

By relying on an environment variable your code doesn't care what a setting is because you can set that at some other point. Actually working with environment variables is really simple. All you need to start is import os and you are off to the races.

Get Environment Variable with Python:

name = os.environ.get('DATABASE_NAME')

The environ part of this call is like a dictionary of all the environment variables. So we just get the one we are after and go about our business. You might even see something like below for your database settings, in your django settings file. You are either getting the environment variable, or you are getting back an empty string.

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql_psycopg2',
        'NAME': os.environ.get('DATABASE_NAME', ''),
        'USER': os.environ.get('DATABASE_USER', ''),
        'PASSWORD': os.environ.get('DATABASE_PASSWORD', ''),
        'HOST': os.environ.get('DATABASE_HOST', ''),
        'PORT': os.environ.get('DATABASE_PORT', ''),
    }
}

Running django-admin.py/manage.py with environment variables

A common use of environment variables in django is the DJANGO_SETTINGS_MODULE. It tells django-admin.py what settings file to load for a project.

DJANGO_SETTINGS_MODULES='project.settings' django-admin.py

manage.py is basically a shortcut to this command. We are setting the location of our settings.py file and running django-admin.py command. There has been a lot debate about manage.py since the above is all it really does. You can read and learn more on a ticket about it: Deprecate and then remove "shell" management command

Set environment variable in bashrc

Finally, depending on your environment you might want to setup environment variables in your bashrc or bash_profile so every time you start a command line environment they are set. To do this add something like the following to your bashrc or bash_profile

export PROGRAMMING_FOLDER='/home/user/programming/'

The export command is built into shell and sets PROGRAMMING_FOLDER to /home/user/programming for the current environment. You can verify this running env in your shell, or even env | grep PROGRAMMING to get just that environment variable.

Common Places to Set Environment Variables in Production

First link is how to set environment variables with supervisord which is a recommended way to run your server side code, generally running gunicorn or uWSGI. uWSGI is linked to because a lot of people run it outside of supervisord. However, it is recommended if you use gunicorn to use supervisord so set environment variables inside that config.

Conclusion

Environment variables are extremely useful, but can be a huge pain to work with. I really suggest playing with them in different environments to get used to working with them. When I first started setting them in production environments it was tough to first learn how to set them, and run my code. However, once I got environment variables working locally in my dev environment, and in production it became super convenient. I am super glad I took the extra effort to learn a process to work with them, and I think you will too.