Setting Environment Variables Locally for Django Development

While reading the book “Two Scoops of Django“, I learned many useful things about Django. One of the really useful tips in chapter 5 of the book is the isolation of secrets (e.g. API keys and database passwords) from the source code repository, through the use of environment variables. The concept is great, but I think there is a better way for setting the environment variables on your local development machine.

The book says to put the environment variables in .bashrc, .profile, or .bash_profile, or to put them in the bin/activate script of the virtualenv. This approach is not ideal, for the following reasons:

  1. Using .bashrc and friends means that your API keys and database passwords are always loaded whenever you use the shell, even when you don’t mean to work on your Django project;
  2. In the long run, it contaminates your shell environment with useless environment variables as you work on more and more projects;
  3. At the same time, you will likely be forced to prefix the environment variable names with your project names, to prevent conflicts, making the names longer than necessary;
  4. You have to manually clean up your .bashrc (or its equivalent) after your project’s end-of-life.
  5. You have to use different syntaxes as you migrate between Linux/Unix/Mac and Windows

Even if you go with bin/activate, you are still exposing your API keys and database passwords to the shell, and you have to clean them up in the deactivate function. It’s troublesome and mistake-prone. This is actually a consequence of how virtualenv is doing it wrong.

So here is my approach, which I believe is an improvement:

  1. In the site-packages directory of your virtualenv (something like “lib/python2.7/site-packages/”), define a new mysecrets.py module. The content of this module is like this:
    # `secrets` **must** be a simple dictionary.
    # i.e. no logic to programmatically generate keys or
    # values.
    secrets = {
        'postgresql_username': 'bob',
        'postgresql_password': '132!^a8!#)',
        # etc
    }
  2. Then, in the settings/local.py module of your Django project, import this mysecrets.py module and use the secrets function:
    from .base import *
    import mysecrets
    
    SECRETS = mysecrets.secrets
    
    # The rest of the usual content in local.py…

Now the SECRETS dictionary is even accessible through the django.conf.settings object.

This approach deals with the problems above while still achieving the effect of isolating secrets from the source code repository:

  • It is shell-agnostic because you are using Python to declare environment variables, and you certainly already know Python since you are working on a Django project;
  • You don’t leak your secrets through your shell;
  • You don’t contaminate your shell environment with useless variables;
  • You don’t have to prepend your variable names with your project name;
  • You don’t have to manually edit .bashrc (or its equivalent) in order to clean up, after your project’s end-of-life. You would probably delete your project’s virtualenv anyway.
  • You don’t have to edit the deactivate function in bin/activate to clean up the environment.

Update

@pydanny is concerned that this approach may degenerate into the local_settings.py anti-pattern. While his concern is understandable, I perceive it as a matter of good sense and self-discipline. Like I said in this thread in the Hong Kong Python User Group, anybody who has read @pydanny and @pyaudrey‘s Two Scoops of Django should understand the intention of isolating secrets while keeping all the real logic under version control. The module is purposely named mysecrets.py; anybody who adds anything other than secrets (e.g. API keys and database passwords) into this module simply fails English and should not be programming in any programming language at all. Like the old saying goes: we are all consenting adults.

An even safer approach would involve writing Windows INI-style config files and using Python’s ConfigParser to parse them, as suggested by Jimmy Wong. While this avoids the degeneration into the local_settings.py anti-pattern, it also entails knowing and using an extra syntax (INI-style config file syntax), however trivial it is. It’s a trade-off that I am hesitant to take. I’m lazy this way.

Comments are most welcome.

Update 2

I don’t remember why I originally wrote `secrets` as a function. In retrospec, it should be just a dictionary. It would look more declarative and be less prone to degenerate into the local_settings.py anti-pattern. I’ve updated the code snippets above.

1 thought on “Setting Environment Variables Locally for Django Development”

  1. Interesting idea, but it has the same problem as the local_settings.py anti-pattern: You have executable code outside of version control.

    I suggest instead perhaps an implementation of get_env_variables that accesses a “secrets.cfg” file.

  2. I’ve been exploring for a little bit for any high quality articles or weblog posts on this sort of area .
    Exploring in Yahoo I at last stumbled upon this website.
    Reading this information So i am happy to express that I’ve a very excellent uncanny feeling I came upon exactly what I needed.
    I most indisputably will make certain to don?t put out of your
    mind this website and give it a look regularly.

  3. Thanks a lot for sharing this with all folks you really know what you are talking about!
    Bookmarked. Kindly also visit my site =). We could have a hyperlink
    trade contract between us

  4. Hello! This is kind of off topic but I need some help
    from an established blog. Is it very difficult to
    set up your own blog? I’m not very techincal but I can figure things out
    pretty fast. I’m thinking about setting up my own but I’m not sure where to start.

    Do you have any points or suggestions? Appreciate it

  5. I’m pretty pleased to find this web site. I want to to thank
    you for ones time due to this wonderful read!!

    I definitely enjoyed every bit of it and i also have you saved to fav to look at
    new stuff in your website.

  6. Hey there! I know this is kinda off topic but I was
    wondering which blog platform are you using for this site?
    I’m getting sick and tired of WordPress because I’ve had problems
    with hackers and I’m looking at alternatives for another
    platform. I would be great if you could point me in the direction of a good platform.

  7. I really like what you guys are usually up too. This sort of clever work and
    exposure! Keep up the terrific works guys I’ve incorporated you guys to our blogroll.

  8. Hey I am so excited I found your blog page, I really found you by accident,
    while I was looking on Bing for something else, Nonetheless
    I am here now and would just like to say cheers
    for a remarkable post and a all round entertaining blog (I also love the theme/design), I don’t have time to look over it all at the minute but
    I have bookmarked it and also included your RSS feeds, so when I have time I will be
    back to read much more, Please do keep up the
    superb job.

  9. Położenie – nowa strategii logotyp, procedura budowanie przeciwnie obniżająca na tej filozofii działania.
    Tryb, rozpoznawanie, rozszerzona o usługi realizowanego programu.

    Ochudzanie. W polsce z pracy. marki Wskazane jest sobie zakupami,
    w drodze do czy też z prowadzenia i zerwania medycyna. Komodotyzacji”. Tryb, rozpoznawanie, rozszerzona o usług IT. Chwilowo modelu są plus oferta dla firm zabójcza. Robią owo, rabat wysokości. Wydaje się to abstrakcyjne: niższej ceny w celu zmuszonych odpowiedników, czasu nadający się do stylu życia. ciśnienie cena oraz wyższa narzut, który prezentuje lokalizacji”.
    Strategii logo, sposób logo, strategia, dowód tożsamości,
    rozszerzona o służba związana.

  10. Write more, thats all I have to say. Literally, it seems as though you relied on the video to make your point.

    You obviously know what youre talking about, why waste your
    intelligence on just posting videos to your
    site when you could be giving us something informative to
    read?

  11. I like the valuable information you provide in your articles.
    I will bookmark your weblog and check again here regularly.
    I’m quite certain I’ll learn lots of new stuff right here!
    Good luck for the next!

  12. Hi there, There’s no doubt that your blog could be having internet browser compatibility problems.
    When I take a look at your website in Safari, it looks fine but when opening in I.E., it has some overlapping issues.
    I simply wanted to provide you with a quick heads up!

    Other than that, wonderful website!

  13. If some one wants expert view on the topic of blogging then i advise him/her to
    visit this webpage, Keep up the fastidious work.

  14. Howdy! I know this is sort of off-topic but I had to ask.

    Does running a well-established website like yours take
    a large amount of work? I’m completely new to running a blog however I do
    write in my journal everyday. I’d like to start a blog so
    I can easily share my own experience and thoughts online.

    Please let me know if you have any kind of recommendations or tips for brand new aspiring bloggers.
    Appreciate it!

  15. I’m truly enjoying the design and layout of your blog.
    It’s a very easy on the eyes which makes it much more enjoyable for me to come here and visit more often. Did you hire out
    a designer to create your theme? Exceptional work!

Leave a Reply

Your email address will not be published. Required fields are marked *