Improving Testing Time at Streema

At Streema we like to develop at a fast pace.  One of the things that allows us to do that is to have an automatically tested codebase. We want to always be shipping and having our apps tested gives us some degree of confidence that we won’t break it all each time we commit a change.

But running tests takes time. A few months ago our unit tests were taking over 12 minutes to run. Each time someone wanted to run the tests, the situation was pretty much like this:

As you might recall, we use Python/Django. And our problem was the use of fixtures in our unit/functional tests. As @carljm said on his “Testing and Django” talk for PyCon 2012, they are slow to load, hard to maintain and increase test interdependence. We had huge fixtures, 1mb fixtures, which loaded thousands of models and they were used on dozens of tests!  With a little bit of patience we were able to remove them all and replace them by using factory boy.

The result:  we reduced the test run time from over 12 minutes to under a minute.  Yes, a single minute.  =)

Stay tuned for more news and insight on the technical details of Streema. We might have an interesting thing or two to share!

Stay tuned!

The Streema Team

How Python and Django power Streema

For a change, this month’s post is from our IT team! We wanted to give you a little sneak peek of the technology that Streema runs on as well as pay due respects to some really well known open source initiatives that make our service possible.

So, we use a ton of open source software*, and while each one has its special role in the Streema universe, there’s a couple we wanted to mention in today’s post. Many of our readers might not be aware of it, but most of our software is built with the Python Programming Language and our web application with the Django Framework. In our experience, they both have proven to be great choices for many reasons, among which we can include a great community, a large amount of available open source libraries and modules, and an elegant and pragmatic design.

Going into more detail, we recently upgraded to Django 1.3 and Python 2.7 as part of an ongoing effort of modernizing our stack and automatizing our processes. When you have a site that’s visited by an interesting amount of users each day (thank you everyone, do keep coming back!), such a change can’t be taken lightly. Thorough testing must be done to correct errors that might spring from the migration. All in all, the migration went pretty smoothly – in big part thanks to how well thought out Python and Django are, but also thanks to a good suite of automated tests we’ve built over time. Though our users might not see any noticeable changes, this upgrade will help us continue making our service better, faster, and stronger than ever.

Thanks again to the vibrant Python and Django communities (and the communities of the rest of the many open source softwares we use).

Stay tuned for more news and insight on the technical details of Streema. We might have an interesting thing or two to share!

————————
* The list is definitely not exhaustive: Apache, Nginx, Munin, MySQL, Redis, Eventlet, Fabric, MongoDB, SQLite, MooTools, Mplayer and of course, WordPress.org for this blog. =)

Powered by Django.

Python Programming Language