Extending Django

Jarrell Waggoner / @malloc47


Online: malloc47.com/posscon2013/

About Me

Ph.D. candidate in the Department of Computer Science and Engineering at the Univeristy of South Carolina


Researcher -○- Software Developer -○- Python Programmer

I do A.I.


https://github.com/malloc47

About Django


Background on Django -○- Demystifying Concepts

What is Django?

  • Not the 1966 or 2012 film
  • Web application framework
  • Written in Python
  • Uses the standard MVC architecture
  • Originally written to manage the Lawrence Journal-World newspaper website
  • Maintained by the Django Software Foundation

Who uses Django?

Major Django users

Django compared with Rails

  • Python vs. Ruby
    • SciPy/NumPy - Science!
    • Linux ubiquity
  • MTV vs. MVC
  • DIY vs. Generators
  • Bundled (django.contrib.*)
    • Admin
    • Authentication
    • Geospatial (GeoDjango)
    • Form handling (previews, wizards, validation, etc.)
  • Optimized for content-heavy CRUD apps

MVC vs. MTV

Model Template View

Model=the data itself
Template=how the data is presented
View=which data is presented

Django Design Philosophy

https://docs.djangoproject.com/en/dev/misc/design-philosophies/

Django Project Structure

Project vs. Apps

  • project/
    • project/
    • app1/
    • app2/
    • app3/
    • ...

Django Files

> django-admin.py startproject myproj

> ./manage.py startapp myapp

  • manage.py
  • myproj/__init__.py
  • myproj/settings.py
  • myproj/urls.py
  • myproj/wsgi.py
  • myapp/__init__.py
  • myapp/models.py
  • myapp/tests.py
  • myapp/views.py

Recommended

  • myproj/templates/myapp/*.html
  • myproj/static/*.{css,js}
Github Search

Installing Requirements

the non-root way


pythonbrew—build/install local copies of Python, independent of the system (forked as pythonz)
virtualenv—local environment for installing python packages
pip—Python package manager
pip install django
requirements.txt—machine-readable list of dependencies for your project
Django==1.4.3
South==0.7.6
scss==0.8.72
simplejson==2.6.1

which can be read by pip

pip install -r requirements.txt

PAAS Support

Django Basics

Request Handling

Request → urls.py → views.py function → <template>.html

GET request for /stuff/

urls.py

urlpatterns = patterns('',
    (r'^stuff/$', views.stuff_handler),
)

views.py

def stuff_handler(request):
    return render(request, 'stuff.html', 
                  {'name': request.user.username})

stuff.html

<html>
<head><title>Page o' Stuff</title></head>
<body>
<p> Your name is {{ name }} </p>
</body></html>

What about Models?

models.py

class Person(models.Model):
    account = models.ForeignKey('django.contrib.auth.models.User')
    name = models.CharField(max_length=64)
    cash = models.BigIntegerField()

views.py

def stuff_handler(request):
    me = Person.objects.filter(account=request.user)
    return render(request, 'stuff.html', {'me': me})

Django Middleware

Default Middleware

MIDDLEWARE_CLASSES = (
    'django.middleware.common.CommonMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
)
Middleware Illustration

Middleware Hooks

class CommonMiddleware(object):
    def __init__():
        pass # only called once
    def process_request(self, request):
        pass
    def process_view(self,request,view_func,view_args,view_kwargs):
        pass
    def process_template_response(self, request, response):
        pass
    def process_response(self, request, response):
        pass
    def process_exception(self, request, exception):
        pass

django.middleware.common.CommonMiddleware

class CommonMiddleware(object):
    def process_request(self, request):
        if 'HTTP_USER_AGENT' in request.META:
            for ua in settings.DISALLOWED_USER_AGENTS:
                if ua.search(request.META['HTTP_USER_AGENT']):
                    logger.warning('Forbidden: %s', request.path,
                        extra={
                            'status_code': 403,
                            'request': request
                        }
                    )
                    return http.HttpResponseForbidden('Forbidden')

django.middleware.common.BrokenLinkEmailsMiddleware

class BrokenLinkEmailsMiddleware(object):
    def process_response(self, request, response):
        if response.status_code == 404 and not settings.DEBUG:
            # sanity checks go here
            ua = request.META.get('HTTP_USER_AGENT', '')
            ip = request.META.get('REMOTE_ADDR', '')
            mail_managers(
                "Broken link on %s" % (request.get_host(),),
                """Referrer: %s
                    Requested URL: %s
                    User agent: %s
                    IP address: %s
                    """ % (request.META.get('HTTP_REFERER', ''),
                           request.get_full_path(), ua, ip),
                fail_silently=True)
        return response

Beautiful Soup

from bs4 import BeautifulSoup
class BeautifulMiddleware(object):
    def process_response(self, request, response):
        if response.status_code == 200:
            if response["content-type"].startswith("text/html"):
                beauty = BeautifulSoup(response.content)
                response.content = beauty.prettify()
        return response

Content

<html><p>Some data<p>Moar data

Beautified

<html>
 <body>
  <p> Some data </p>
  <p> Moar data </p>
 </body>
</html>

Content

Pure text

Beautified

<html>
 <body>
  <p>Pure text</p>
 </body>
</html>

http://pyevolve.sourceforge.net/wordpress/?p=814

Custom Fields

Why Custom Fields?

  • Obscure column types (e.g. geographic data type)
  • Custom column types (e.g. PostgreSQL custom types)
  • Non-builtin Python type to serialize into DB table

Set Default Attributes

django_extensions.db.fields.CreationDateTimeField

class CreationDateTimeField(DateTimeField):
    def __init__(self, *args, **kwargs):
        kwargs.setdefault('editable', False)
        kwargs.setdefault('blank', True)
        kwargs.setdefault('default', datetime.now)
        DateTimeField.__init__(self, *args, **kwargs)

instead of

class Person(models.Model):
    create = DateTimeField(editable=False, 
                           blank=True, 
                           default=datetime.now)

we get

class Person(models.Model):
    create = CreationDateTimeField()

github.com/django-extensions/django-extensions/

Add New Model Attributes

expirefield.fields.ExpireField

class ExpireField(DateTimeField):
    def __init__(self, verbose_name=None, name=None, **kwargs):
        # get attribute
        self.duration = kwargs.pop('duration')
        # check type
        if not isinstance(self.duration, timedelta):
            raise FieldError
        super(ExpireField, self).__init__(verbose_name, 
                                          name, 
                                          **kwargs)

allows

class Person(models.Model):
    expire_time = ExpireField( duration=timedelta(hours=2) )

This snippet does nothing... yet

github.com/malloc47/expirefield

Modifying Parent Model

django_extensions.db.fields.UUIDField

class UUIDField(CharField):
    def __init__(self, verbose_name=None, name=None, **kwargs):
        kwargs.setdefault('max_length', 36)
        self.empty_strings_allowed = False
        kwargs['blank'] = True
        kwargs.setdefault('editable', False)
        CharField.__init__(self, verbose_name, name, **kwargs)

    def contribute_to_class(self, cls, name):
        if self.primary_key:
            assert not cls._meta.has_auto_field, "More than one PK"
            super(UUIDField, self).contribute_to_class(cls, name)
            cls._meta.has_auto_field = True
            cls._meta.auto_field = self
        else:
            super(UUIDField, self).contribute_to_class(cls, name)

    def pre_save(self, model_instance, add):
        value = super(UUIDField, self).pre_save(model_instance,add)
        if add:
            value = force_unicode(uuid.uuid4())
            setattr(model_instance, self.attname, value)
        return value

    def get_internal_type(self):
        return CharField.__name__

github.com/django-extensions/django-extensions/

Specialized Serialization

Pickling Objects

class PickledObjectField(models.Field):
    
    def __init__(self, *args, **kwargs):
        kwargs.setdefault('null', True)
        kwargs.setdefault('editable', False)
        super(PickledObjectField, self).__init__(*args, **kwargs)

    def to_python(self, value):
        if value is not None:
            value = pickle.loads(b64decode(value))
        return value

    def get_db_prep_value(self, value):
        if value is not None:
            value = force_unicode(b64encode(pickle.dumps(value)))
        return value

    def get_db_prep_lookup(self, lookup_type, value):
        if lookup_type not in ['exact', 'in', 'isnull']:
            raise TypeError('Lookup type %s is not supported.' % lookup_type)
        return super(PickledObjectField, self).get_db_prep_lookup(lookup_type, value)
    def value_to_string(self, obj):
        value = self._get_val_from_obj(obj)
        return self.get_db_prep_value(value)

    def get_internal_type(self): 
        return 'TextField'

    __metaclass__ = models.SubfieldBase # to_python will be called

http://djangosnippets.org/snippets/1694/

Management Commands

manage.py ⇒ django-admin.py

./manage startapp
./manage syncdb
./manage shell
./manage runserver localhost:8000
./manage changepassword
./manage help

New commands:

appname/management/commands/command_name.py

Why Management Commands?

  • Create/Process data (fixtures might be better)
  • Override default command behavior
  • Expose functions to run as cron jobs

Model Introspection

expirefield.commands.management.expire

Remember ExpireField

class Command(BaseCommand):
    def handle(self, *args, **options):
        # get all models that have an ExpireField
        models = [m for m in django.db.models.get_models() 
             if any(type(f) is ExpireField for f in m._meta.fields)]
        for model in models:
            field = next(f for f in model._meta.fields 
                        if type(f) is ExpireField )
            filter_args = {'{0}__{1}'.format(field.name, 'lt'): 
                           (datetime.now() - field.duration),}
            # delete all the old objects
            q = model.objects.filter(**filter_args)
            q.delete()
./manage expire

github.com/malloc47/expirefield

optparse Arguments

class Command(BaseRunserverCommand):
    option_list = BaseRunserverCommand.option_list + (
        make_option('--sass',
            action='store_true',
            dest='sass',
            default=False,
            help='process all sass files in project'),
        )

    def inner_run(self, *args, **options):
        if options['sass']:
            print('Processing SASS files...')
            files = find_scss('assets/')
            for f in files:
                print('==> Converting ' + f)
                with open(os.path.splitext(f)[0]+'.css','w') as h:
                    h.write(parser.load(f))

        # continue with the runserver command
        super(Command, self).inner_run(*args, **options)

def find_scss(top):
    return reduce(
        list.__add__,
        [[os.path.join(root, f)
          for f in files
          if f.endswith('.scss') or f.endswith('.sass')]
         for root, dirs, files
         in os.walk(top)],
        []) 
./manage runserver --sass

Fun Django Things

importd — http://pythonhosted.org/importd/

from importd import d # the d is for Django

@d("/")
def index(request):
    return d.HttpResponse("hello world")

jython — https://code.google.com/p/django-jython/

jython manage.py war

cheat sheet — http://media.revsys.com/images/django-1.4-cheatsheet.pdfs

debug toolbar — https://github.com/django-debug-toolbar/django-debug-toolbar

Questions?