Nick Carroll, PhD

I was sifting through my blog entries during the holidays and came across my post three years ago about my first month at ThoughtWorks. After reading it I felt it needed an update, otherwise I probably wouldn’t have left the company if the list was still accurate. My last day at ThoughtWorks was 24th December 2009.

1. Other ThoughtWorkers: I have had the chance to work with some of the most incredible software developers that I have ever met. These guys and gals really know how to thrash a keyboard around when pumping out quality code. It has to be said that working with really talented individuals makes work very enjoyable. Which goes to show that Roy’s social experiment is still going strong after all these years.

The people at ThoughtWorks are certainly a great bunch of people. I would still rank working with many of the fine folks at ThoughtWorks as one of my best experiences. However, many of them have since moved on to greener pastures.

2. Ruby is such a cool programming language and I am so glad that I am working at a company that has completely embraced it. Ruby has been around in the US and Europe for a while, but it is still relatively new in Australia. I am waiting impatiently for the Aussie tech industry to catch up so that I can finally work on a Ruby project.

Meanwhile I have been ramping up on my Ruby skills, and there is no better place in Australia for learning Ruby. We have some of the best Ruby developers working in Sydney at the new ThoughtWorks Studios, and they open their doors after work each week for us to work on a Ruby project with them.

Sadly, I never got to write a line of Ruby code that made it into production. It was only ever Java code. Also, ThoughtWorks Studios is no longer based in Sydney.

3. Agile is certainly an interesting approach to developing software. I have already been thrown head first into a large agile software development project, and I am already hooked on the agile koolaid. Over the weekend our client deployed the first release in time for a public launch of their system. It is the first large scale project that I have worked on, and it was a success right off the bat. The release was completed on-time, on-budget, and with 5 times less defects than any of their previous projects. Being part of a team that produced a result like this certainly made my first month a memorable one.

I am still a strong advocate for Agile software development. I was never part of a project that failed to deliver, and I am extremely proud of this success.

4. Open Communication: I have been amazed with how open the lines of communication are within the company. In the first two weeks I had met Martin Fowler our lead scientist, coffee with Bruce the Australian MD, and beers with Roy Singham the founder of the company. They all made themselves available to talk about agile, corporate strategy, and whatever the next big thing might be. I thought it was cool to be able to hang out with these guys. There aren’t many companies out there that have corporate leaders that like to hang out with their employees and have a general interest in them.

Communication is not as open as it used to be.

5. Geek Night is just one of many ThoughtWorks events that helps to facilitate knowledge sharing within the company. It is an after work event where developers head back to the ThoughtWorks office for a night of geeking out in front of a computer. We learn a new programming language whilst downing copious amounts of Coke Zero and rice cracker snacks.

Geek Night rarely happens. I tried my best to inject some life into it with the Functional Programming and Groovy user groups, but the interest just wasn’t there.

6. Free food: I have put on a couple of kilos since I started work. It has to do with all the lunches and dinners that the company has put on. They keep a fridge stocked with beers, wine, soft drinks and fruit juice. As well as bowls of fruit and cupboards filled with biscuits, chips, and chocolates. Best of all they have a tab going for free coffee at a local cafe!

There is still free food on Fridays. The coffee tab at the local cafe got replaced with an in-house coffee machine.

7. Consulting Dojo: There is one catch to the free food, on Fridays lunch is provided to attract all the ThoughtWorkers to the office to listen to one of our colleagues present on a consulting related topic. The presentations have been very interesting, and there is always a good turn out. The dojo is another example of knowledge sharing within the company.

The dojos are still around, but in a lighter form. A couple of presenters are given 5 minutes to present on a topic that ranges from cool stuff that happened on a project to how to use a Mac.

8. Training and book budgets: Each ThoughtWorker is provided with their own personal budget for training and purchasing books. Essentially we are responsible for our own learning and personal development. So we can choose whatever course or conference we want to attend. There is just so much support for our career development.

The training and book budgets are still around. There was a period when the training budget was reduced, but it is back to where it was. I only ever got to use my training budget in my first year, after that I found it hard to find time for training as I was always on a project and focused on delivery. However I made good use of the book budget. It is something that I will certainly miss.

9. Brand new Dell Latitude D620: It isn’t a Mac, but I have been very impressed with this piece of hardware. I really like the widescreen display, Intel Core Duo processor, and the feel of the keyboard. It also doesn’t get that hot from extended usage. It is the first Dell I’ve used, and I’m very happy with it.

Would you believe that the Macbook Pro is now the default laptop? It was a sad day when I had to return my new 15 inch Macbook.

10. Mobile phone and home broadband: This is a nice perk, having your company pay for your mobile phone and home broadband bills. I can finally afford ADSL2!

You still get a mobile phone and broadband budget. So you can sign up for an iPhone when you join.

This will be my last post on ThoughtWorks as I close this chapter of my life.

My Estimation Deck iPhone application just got published in the iTunes store! You can download it for free and use it for your next Agile estimation session. It saves me from carrying around a deck of cards, and I hope it saves you from that pain as well!

I have been working as part of a team at ThoughtWorks to build an iPhone app for the upcoming Agile Australia 2009 conference. It has been an anxious wait for Apple to approve the app, but it was a thrill when that email finally came through. I am pleased to announce that the Agile Australia 2009 iphone app is now available for download at the iTunes store for free! If you are attending the conference then this companion app will keep you informed of the schedule, provide feedback, follow the hot sessions, find your rooms, and tweet about the talks.

Hoorah! Just got my new work laptop. It is a brand spanking new 15 inch Macbook pro! Apparently it is the new default laptop for developers at ThoughtWorks Australia.

This guide is basically a rehash of this posting in Joyent’s support forums. I am reproducing it below to include my experiences as I found some discrepancies with the posting in the Joyent forums.

Joyent Shared Accelerators don’t allow you to deploy your Django app using mod_python, so you have to create a proxy path that diverts traffic to lighttpd and FastCGI to serve your Django app.

For this guide you will need to replace ${USER} with your username on Joyent’s server, the hostname of your server as ${HOST}, and your DNS domain as ${DOMAIN}.

Set up Apache as a proxy server for lighttpd

You no longer need to submit a support ticket to request a port number for lighttpd. You can just go to Virtualmin for your server (https://virtualmin.joyent.us/${HOST}/) > Other Tools > Check ports to view a list of available port numbers that have been reserved for you. Pick one and note it down. We will refer to this port number as ${PORT}.

Set up a directory structure for lighttpd:

mkdir -p ~/etc/init.d
mkdir -p ~/etc/lighttpd/vhosts.d
touch ~/logs/lighttpd.error.log ~/logs/lighttpd.access.log

Using a text editor, create ~/etc/lighttpd/lighttpd.conf:

#-- Lighttpd modules

server.modules = ( "mod_rewrite",
                                "mod_redirect",
                                "mod_access",
                                "mod_cgi",
                                "mod_fastcgi",
                                "mod_compress",
                                "mod_accesslog",
                                "mod_alias" )

#-- Fundamental process configs
server.port = ${PORT}
server.username = "${USER}"
server.groupname = server.username
var.base = "/users/home/" + server.username
server.pid-file = base + "/var/run/lighttpd.pid"

#-- Logging
server.errorlog = base + "/logs/lighttpd.error.log"
accesslog.filename = base + "/logs/lighttpd.access.log"

#-- Default
server.document-root = base + "/web/public"
server.indexfiles = ( "index.php", "index.html",  "index.htm", "default.htm" )

#-- Security
url.access-deny = ( "~", ".inc", ".ht" )

#-- Mimetypes
include_shell "cat " + base + "/etc/lighttpd_mimetypes.conf"

#-- VHOSTS

Create ~/etc/lighttpd/mimetypes.conf:

mimetype.assign             = (
".pdf"          =>      "application/pdf",
".sig"          =>      "application/pgp-signature",
".spl"          =>      "application/futuresplash",
".class"        =>      "application/octet-stream",
".ps"           =>      "application/postscript",
".torrent"      =>      "application/x-bittorrent",
".dvi"          =>      "application/x-dvi",
".gz"           =>      "application/x-gzip",
".pac"          =>      "application/x-ns-proxy-autoconfig",
".swf"          =>      "application/x-shockwave-flash",
".tar.gz"       =>      "application/x-tgz",
".tgz"          =>      "application/x-tgz",
".tar"          =>      "application/x-tar",
".zip"          =>      "application/zip",
".mp3"          =>      "audio/mpeg",
".m3u"          =>      "audio/x-mpegurl",
".wma"          =>      "audio/x-ms-wma",
".wax"          =>      "audio/x-ms-wax",
".ogg"          =>      "audio/x-wav",
".wav"          =>      "audio/x-wav",
".gif"          =>      "image/gif",
".jpg"          =>      "image/jpeg",
".jpeg"         =>      "image/jpeg",
".png"          =>      "image/png",
".xbm"          =>      "image/x-xbitmap",
".xpm"          =>      "image/x-xpixmap",
".xwd"          =>      "image/x-xwindowdump",
".css"          =>      "text/css",
".html"         =>      "text/html",
".htm"          =>      "text/html",
".js"           =>      "text/javascript",
".asc"          =>      "text/plain",
".c"            =>      "text/plain",
".conf"         =>      "text/plain",
".text"         =>      "text/plain",
".txt"          =>      "text/plain",
".dtd"          =>      "text/xml",
".xml"          =>      "text/xml",
".mpeg"         =>      "video/mpeg",
".mpg"          =>      "video/mpeg",
".mov"          =>      "video/quicktime",
".qt"           =>      "video/quicktime",
".avi"          =>      "video/x-msvideo",
".asf"          =>      "video/x-ms-asf",
".asx"          =>      "video/x-ms-asf",
".wmv"          =>      "video/x-ms-wmv",
".bz2"          =>      "application/x-bzip",
".tbz"          =>      "application/x-bzip-compressed-tar",
".tar.bz2"      =>      "application/x-bzip-compressed-tar"
)

Finally, create an init script at ~/etc/init.d/lighttpd:

#!/bin/sh

HOME=/users/home/${USER}
LIGHTTPD_CONF=$HOME/etc/lighttpd/lighttpd.conf
PIDFILE=$HOME/var/run/lighttpd.pid

case "$1" in

    start)
    # Starts the lighttpd daemon
    echo "Starting lighttpd"
    PATH=$PATH:/usr/local/bin /usr/local/sbin/lighttpd -f $LIGHTTPD_CONF

;;
    stop)
    # stops the daemon bt cat'ing the pidfile
    echo "Stopping lighttpd"
    kill `/bin/cat $PIDFILE`

;;
    restart)
    ## Stop the service regardless of whether it was
    ## running or not, start it again.
    echo "Restarting lighttpd"
    $0 stop
    $0 start

;;
    reload)
    # reloads the config file by sending HUP
    echo "Reloading config"
    kill -HUP `/bin/cat $PIDFILE`

;;
    *)
    echo "Usage: lighttpd (start|stop|restart|reload)"
    exit 1
;;
esac

Don’t forget to make the init script executable:

chmod 755 ~/etc/init.d/lighttpd

Proxy Apache to lighttpd

Open up a web browser, and log into https://virtualmin.joyent.us/${HOST}/

Select the virtual server to configure. Then go to Server Configuration > Proxy Paths > Add a new proxy path. Enter the following values and click Create.

Local URL path: /
Destination URLs: http://${DOMAIN}:${PORT}

Configure Django environment

Add the path to your Django app to the PYTHONPATH. Add the following to your .profile and .bashrc files.

export PYTHONPATH=/users/home/${USER}/src/django_projects

Deploying your Django app

Check out your django app to /users/home/${USER}/src/django_projects. I will refer to this django app as ${APPNAME}.

cd ~/src/django_projects
svn co svn+ssh://subversion_repos/site/${APPNAME}/trunk ${APPNAME}

Create a MySQL database

Normally I would use PostgreSQL cos it rocks, but unfortunately Joyent only provides database restrictions for specific users on MySQL. So we’ll create a mysql user and grant it privileges to access a mysql database.

Open up a web browser and log into https://virtualmin.joyent.us/${HOST}/

Select ${DOMAIN} from the dropdown list > click “Edit Databases” > Click “Create a new database”.

I entered “production” into the Database name field so that my database will be called ${USER}_${DOMAIN}_production. Then click “Create”.

Create a database user

In Virtualmin, Select ${DOMAIN} from the dropdown list.
Click “Edit Mail and FTP Users” > “Add a user to this server”.
Under Virtual domain user mailbox details, enter “django” into the Email address field. This will create a mysql user called django-${DOMAIN}, and the auto-generated password will also be used for the mysql password in your django settings.py.

Expand “Quota and home directory settings”. Limit the user’s home directory quota to 1MB.
Expand “Other user permissions”. Allow the user access to the “${USER}_${DOMAIN}_production” database we just created. Click create.

Configure project settings

In settings.py modify your database settings to the following:

FORCE_SCRIPT_NAME=''

import os.path
ROOT_DIR = os.path.abspath(os.path.dirname(file))

DATABASE_ENGINE = 'mysql'
DATABASE_NAME = '${USER}_${DOMAIN}_production'
DATABASE_USER = 'django-${DOMAIN}'
DATABASE_PASSWORD = 'password"
DATABASE_HOST = ''
DATABASE_PORT = ''

MEDIA_ROOT = os.path.join(ROOT_DIR, 'media')
MEDIA_URL = '/media/'
ADMIN_MEDIA_PREFIX = '/media/admin/'

TEMPLATE_DIRS = (
    os.path.join(ROOT_DIR, 'templates'),
)

INSTALLED_APPS = (
    'django.contrib.sites',
    'django.contrib.admin',
    'django.contrib.flatpages',
)

Since the settings file has our MySQL password inside, don’t let others read it:

chmod 600 ~/src/django_projects/${APPNAME}/settings.py

Then create the database tables in the usual fashion:

./manage.py syncdb

Create project init script

Create ~/src/djangoprojects/${APPNAME}/etc/init.sh:

#!/bin/sh

HOME="/users/home/${USER}" # Edit to your own username
PYTHONPATH=$HOME/src/django_projects
export PYTHONPATH

PROJECT_NAME="${APPNAME}"
PROJECT_DIR="$HOME/src/django_projects/$PROJECT_NAME"
PID_FILE="$HOME/var/run/$PROJECT_NAME.pid"
SOCKET_FILE="$HOME/tmp/$PROJECT_NAME.socket"
MANAGE_FILE="$PROJECT_DIR/manage.py"
METHOD="prefork"

case "$1" in

    start)
    # Starts the Django process
    echo "Starting Django project $PROJECT_NAME"
    python $MANAGE_FILE runfcgi maxchildren=2 maxspare=2 minspare=1 method=$METHOD socket=$SOCKET_FILE pidfile=$PID_FILE

;;
    stop)
    # stops the daemon by cat'ing the pidfile
    echo "Stopping Django project $PROJECT_NAME"
    kill `/bin/cat $PID_FILE`

;;
    restart)
    ## Stop the service regardless of whether it was
    ## running or not, start it again.
    echo "Restarting Django project $PROJECT_NAME"
    $0 stop
    $0 start

;;
    *)
    echo "Usage: init.sh (start|stop|restart)"
    exit 1

;;
esac

Make the init script executable:

chmod 755 ~/src/djangoprojects/${APPNAME}/etc/init.sh

Offload static media to lighttpd

We don’t want Django to be serving static content, so any path that refers to static content will be served by the web server directly from ~/web/public.

Create a softlink from django’s admin media to ~/web/public/media/admin.

mkdir ~/web/public/media
ln -s /usr/local/lib/python2.5/site-packages/django/contrib/admin/media/ ~/web/public/media/admin

Create a softlink from your project’s media directory to ~/web/public/media/public.

mkdir -p ~/src/django_projects/project/media/public
ln -s /users/home/${USER}/src/django_projects/${APPNAME}/media/public ~/web/public/media/public

Configure lighttpd

Edit ~/etc/lighttpd/vhosts.d/${APPNAME}.conf.

$HTTP["host"] =~ "(www.)?${DOMAIN}" {
    server.document-root = base + "/web/public"
    fastcgi.server = (
        "/${APPNAME}.fcgi" => (
            "main" => (
                "socket" => base + "/tmp/${APPNAME}.socket",
                "bin-environment" =>
                            ( "TZ" => "America/Chicago" ),
                "check-local" => "disable",
            )
        ),
    )

    url.rewrite-once = (
        "^(/media/admin.*)$" => "$1",
        "^(/media/public.*)$" => "$1",
        "^/favicon.ico$" => "/media/public/img/favicon.ico",
        "^(/.*)$" => "/${APPNAME}.fcgi$1",
    )
}

Then include vhosts.d/${APPNAME}.conf in your lighttpd.conf:

echo 'include "vhosts.d/${APPNAME}.conf"' >> ~/etc/lighttpd/lighttpd.conf

Schedule service start

Create a Joyent bootup action in Virtualmin. In Virtualmin, Select ${DOMAIN} from the dropdown list > go to Services > Booup Actions > Add a new bootup action. Enter the following values in the input fields and click “Create”.

Action name: init-${APPNAME}-django-site
Description: Init ${APPNAME} Django Site
Commands to run at startup: /users/home/${USER}/src/django_projects/${APPDNAME}/etc/init.sh start

Go back to the Bootup Actions, click “Add lighttpd”. Enter the following values in the input fields and click “Create”.

Action name: lighttpd-${APPNAME}-django-site
Description: Lighttpd ${APPNAME} Django Site
Commands to run at startup: /usr/local/sbin/lighttpd -f /users/home/${USER}/etc/lighttpd/lighttpd.conf

Open up your browser and go to your newly deployed Django app!

If you don’t see your site then you will have to do some debugging. I didn’t get it first time as you’ll note that my instructions above are slightly different from the original post here.

I wish I never discovered Django templates because I shudder every time I have to use Freemarker or Velocity for my Spring MVC applications. Fortunately there is an alternative. You can use StringTemplate to generate your views in Spring MVC quite easily. StringTemplate enforces a strict separation of concerns, which therefore minimises the amount of logic that is allowed in the view, thus forcing you to do more in your controllers. Therefore your view remains purely for presentation purposes.

First things first is that you need to download StringTemplate and Antlr, and make those libraries available on your classpath. Next, extend Spring’s InternalResourceView.

import org.springframework.web.servlet.view.InternalResourceView;
import org.springframework.core.io.Resource;
import org.antlr.stringtemplate.StringTemplateGroup;
import org.antlr.stringtemplate.StringTemplate;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.util.Map;
import java.io.PrintWriter;

public class StringTemplateView extends InternalResourceView {

    @Override
    protected void renderMergedOutputModel(Map model, HttpServletRequest request,
                HttpServletResponse response) throws Exception {

        Resource templateFile = getApplicationContext().getResource(getUrl());

        StringTemplateGroup group = new StringTemplateGroup("webpages", templateFile.getFile().getParent());
        StringTemplate template = group.getInstanceOf(getBeanName());
        template.setAttributes(model);

        PrintWriter writer = response.getWriter();
        writer.print(template);
        writer.flush();
        writer.close();
    }
}

Finally, you need to configure your Spring application context to use StringTemplate to render your views.

Now you can organise your StringTemplate files that have the “.st” suffix in /WEB-INF/templates. For example, you might want to organise your templates as follows.

WEB-INF
        + templates
                + layout
                        layout.st
                + partials
                        header.st
                        footer.st
                        feedback_form.st
                contact_us.st

The layout.st template might look like the following template.

<html>
    <body>
$partials/header()$
$body$
$partials/footer()$

    </body>
</html>

The above layout.st contains the basic structure for an HTML page on your site. You can use this one template to keep a consistent look and feel across your entire site. The layout.st template depends on the header.st and footer.st templates that exist in the partials directory.

To insert the feedback form in feedback_form.st into the $body$ placeholder attribute for your contacts page you simply create a contact_us.st template with the following line.

$layout/layout(body=partials/feedback_form())$

When your controller handles the request to display the contact us page, the controller will return a ModelAndView with the view name set to “contact_us”. The view name maps to the contact_us.st file in your templates directory. StringTemplate will render the contact_us page using the layout template with the header, footer and feedback form templates.

It makes me very happy to see others championing continuous deployment. Apparently the developers at IMVU release into production a number of times a day, as often as 50 releases in a day. Smells like a lot of bug fixing than new features, but at the very least they have a lean process for releasing into production. I believe having the capability to release into production as often as possible is more important than having a process that draws out the length of time that it takes to release into production.

I’ve always been a fan of getting code into production as quickly as possible. Releasing into production early and often adds business value and keeps users happy. Adding new functionality that makes it easier to add items to a shopping cart for example means the business can encourage more fluid sales through a better online experience. It also means certain bugs won’t provide a detrimental user experience that would drive customers away. You will drive more customers away the longer a bug stays in production. Or if you are lucky, have one of your users get so ticked off with your application that they provide their own patch and release it for you.

My preferred approach to continuous deployment is to integrate it as part of your continuous integration (CI) build box. With Cruise or Bamboo create a build that runs an ant script that creates a branch, compiles source, runs tests, and deploys on a successful build. You might want to keep this build separate from your regular trunk builds, as you may not want to release every time someone checks code into the source code repository, especially when your team gets into a healthy habit of doing atomic commits.

Utilising your CI box in your deployment process effectively means you can release into production with one click of a button. Imagine that, one click deployment, one click that automates your quality assurance process (assuming you have a quality suite of unit, integration, and functional tests), and production artifacts such as WAR files and configuration properties for archiving. Spending time improving your deployment process and making it leaner means you can realise business value earlier and keep your customers happier.

Isaiah, a friend from uni days and colleague at ThoughtWorks published a good article at MSDN magazine on using mocks and tests to design role-based objects.

Holy crap! I was just perusing the bin directory of my JDK installation and stumbled on a tool called VisualVM. It is a free lightweight profiling tool for the JVM. I can easily find which Java process maps to a specific Java application. Better yet the tool tells me what the PIDs are so that I can easily terminate “rogue” Java processes. VisualVM also provides visualisation tools for monitoring Java Classes, Threads, PermGen, and the Heap. It saves me from having to buy JProfiler or setting up Netbeans to profile my Java applications.

Last night at the Groovy Group we had Graeme Roche from SpringSource and lead developer of Grails give a webinar titled “Building Twitter with Grails in 40 minutes“. We were all impressed, not only with how Grails leverages Spring and third party plugins but also with the Twitter application that Graeme was building. He gave a new meaning to TDD — Twitter Driven Development. Each new feature that he added allowed him to tweet about it in the application that he was building. It was evident that he enjoys talking about Grails and connecting with user groups, even the flu wasn’t going to keep him away from giving the talk!