Where can I learn about managing a robust deployment process between dev and prod environments?

Ryan asked:

I am currently editing my Python code (Flask Web Server) on my live production server. I’m just getting a website started and understand this is not a viable long-term solution. I understand that many people have many different approaches for managing a development, a testing, and a production environment (and probably more), but can someone please point me to a place where can I can get started learning more about managing this type of process flow? I don’t know where to begin. Thanks.

My answer:

Let’s stop for a minute and recall why we separate development and production environments in the first place.

In short, it’s because we are human and make mistakes. The point of having separate environments is to catch and correct mistakes as early as possible, and before they impact our users.

There are almost as many ways to set this up as there are applications to be deployed. Whatever you are doing, your best bet is to use a methodology which is simple enough to understand and use that you aren’t tempted to work around it, and correct enough to help you catch problems before they reach production.

Now if you want to be enterprisey about it, and set up something that almost nobody understands in its entirety and almost nobody actually likes, then by all means follow ITIL. There’s enough process in there to keep you buried in paperwork for half your career. Though, you may have to do this if you are developing software in certain industries or for certain clients.

From your question, though, it sounds like you are a single developer or a small team, working on a project which needs a bit more flexibility and speed.

In essence you need two types of tools: testing tools and deployment tools.

At minimum, all you really need is two servers: the production server, and a testing (staging) server. You’ll first test your changes on your local workstation. If everything seems all right, you push your code to the staging server, and test it again. If everything still seems OK, you push it to production and wait for the fire alarm to go off.

One of the most powerful methodologies for testing your software is continuous integration. With this method, whenever you check in code to a shared repository, a server checks out the code and runs your entire test suite. (You did write unit and functional tests for everything, right?) It can also do integration testing. If all the tests pass, you can configure such tools to promote the code to production and deploy it automatically. Github does this, for instance. Two common CI tools today are Jenkins and Travis.

Finally there’s deployment. This seems obvious: you copy your new code to the (staging or production) server and then tell the web server to start new workers on the new code. Tools such as Capistrano, Commando or Deployer help automate this. They can also typically roll back to a previous deployment if something goes wrong. (And your language or framework should have ways to deal with rolling back database schema changes, such as migrations in Rails. For Flask, you might use something like SQLAlchemy and Alembic.)

The end state you probably want to be in is one where you have 100% test coverage, with unit, functional and integration tests for everything, so that you can take advantage of continuous integration to deploy your changes — or catch mistakes — as rapidly as possible. Start small, with a single “testing” server, and work your way up, until you get there.

View the full question and any other answers on Server Fault.

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.