Articles

For any developer—regardless of seniority—estimating the time you think a development task or project will take can be a daunting task. I was reminded of this again recently after I read software engineer @jaimeohm's heavily re-tweeted tweet about what makes "us coders" special: "We are expected to know how to do things we've never done before and estimate how long they will take." Where seniority becomes a factor is having more experience and being able to more accurately guess how long a thing will take. Of course, this isn't an issue unique to software engineering, but the problem is that people without this experience (CEOs, Project Managers, n00bs, et al [I'm generalizing here]) often just think that a coding task is just opening up a text editor and putting some text in there. It's never quite that simple: It's thinking about scale, maintenance, communicating with teams and clients, installing software, setting up environments, research, discussion with designers, user experience—all that comes with creating a product. In short, you need to estimate for the entire project and not just the "writing code" part of it.

With this in mind, it's important to not only pad your time estimates a bit, but also communicate immediately with your team or managers when a task is proving more difficult than estimated, i.e. you estimated incorrectly. You cannot worry about this effect; it will happen. You should plan for it to happen and managers should not get upset when an estimate falls short. Once you're actually working on the task, the absolute worst thing you can do is to sit frustrated with a bug or issue and not enlist the help of fellow team members or keep your team aware that you are stagnating. We all know the experience of spending hours on a tricky problem, only to solve it in 5 minutes with a fresh pair of eyes after a good night's sleep. Having a buddy take a look at your problem can often solve that immediately. Ask questions; lots of them. Absorb everyone around you's knowledge. If someone knows about a library you've been asked to use, ask them for tips. Typically, most developers understand the pain and will happy to help you estimate (seriously, hit me up on AIM any time if you want to chat).

Now, padding time might sound like a negative thing (especially to your clients), but, in the end, you're setting a budget expectation, so when you do end up completing that feature more quickly than expected, your producer is happy, your client is happy and you've potentially made your company some profit. Don't forget than some features will take longer than expected so now you may have time to work on another feature that you underestimated or add that really kick-ass feature to surprise and delight your team and client.

I thought I'd share a recent example of how coding projects are often less about code and more about the things you don't estimate and the things most people don't see. Below you'll find a sample from a project log I wrote during the development of a Ruby on Rails application. I had done some Rails when it was version ~0.9, but had not touched it in it's modern incarnation (which was 3.0.9 when this was written), so I was basically starting from scratch. For the budget, I estimated based on the time that something like this would have taken me to produce in PHP (the language I use the most frequently) and added a bit of extra time for the unknown. You can see in this log that 90% of the work I'm doing has absolutely nothing to do with writing the code. I of course didn't expect to run into *any* of the issues in the log since I had never done it before, but now with this experience, I know to factor in some extra time to get my environments all ready to actually get to the "writing code" stage or I can estimate lower since I know how it works now.

gem update --system

gem update

rails new app --database=mysql

bundle install

sudo gem install mysql2

gem install mysql2 -- --with-mysql-config=/Applications/MAMP/Library/bin/mysql_config

brew install mysql

/usr/local/Cellar/mysql/5.5.14

gem install mysql2 -- --with-mysql-config=/usr/local/Cellar/mysql/5.5.14/bin/mysql_config
Building native extensions.  This could take a while...
Successfully installed mysql2-0.3.6
1 gem installed

bundle install

rake db:create

socket: /Applications/MAMP/tmp/mysql/mysql.sock

brew install mongodb

gem 'bson_ext', '~> 1.3'
gem 'mongoid', '~> 2.1'

rails generate mongoid:config

require 'active_record/connection_adapters/mysql2_adapter'
require 'active_record/connection_adapters/postgresql_adapter'

ActiveRecord::ConnectionAdapters::Mysql2Adapter::NATIVE_DATABASE_TYPES[:big_primary_key] = "BIGINT UNSIGNED NOT NULL AUTO_INCREMENT PRIMARY KEY".freeze
ActiveRecord::ConnectionAdapters::PostgreSQLAdapter::NATIVE_DATABASE_TYPES[:big_primary_key] = "bigserial primary key".freeze

# In order to properly set up single collection inheritance,
# Mongoid needs to preload all models before every request in development mode.
config.mongoid.preload_models = true if Rails.env.development?

# mongoid has overridden activerecord as the default orm so any attempts
# to generate migrations or ActiveRecord models will fail. Therefore we
# must configure our default generators so we can still use activerecord.
config.generators do |g|
  g.orm :active_record
end

namespace :db do

  desc 'Drop and recreate the database and reseed it.'
  task :reboot => [:drop, :create, :migrate, :seed] do
    puts 'Database rebooted.'
  end

  desc 'Recreate the database with a blank slate.'
  task :clean => [:drop, :create, :migrate] do
    puts 'Database cleaned.'
  end

end

How do you estimate? Let's talk about it on Twitter @typeoneerror.

Working on a fantastic project lately; a PHP platform developed with Smarty template engine that will power three (and more) websites. Each website has a shared library used to bootstrap the app and the site-specific logic is contained in the extensible smarty templates.

Each site has a large body of "documentation" files. The client's previous situation was such that each person responsible for the documentation was using their own (and thus, different) solution. So I recommended settling on a standardized documentation format.

I thought Markdown was a great fit and found a fantastic PHP port of John Gruber's tool. This particular version—Markdown Extra—extends the Markdown port to footnotes, tables, and other nice things. I created a very simple smarty plugin (the initial version below) for documentation writers to be able to load in a markdown file into a Smarty Template:

// smarty/plugins/block.markdown.php

function smarty_block_markdown($params, $content, &$template, &$repeat)
{
    if (!$repeat)
    {
        if (isset($params['source']))
        {
            $source = $params['source'];
            $contents = file_get_contents($source);
            $rendered = Markdown($contents);
        }
        else if (!empty($content))
        {
            $rendered = Markdown($content);
        }
        else
        {
            return "";
        }

        return $rendered;
    }
}

This is quite a simplified version of the final plugin. The final includes support for replaceable tokens among other things, but basically, it's used like so:

{* render the contents of a file called markdown/file/name.md as HTML *}
{markdown source="markdown/file/name"}{/markdown}

{* render the markdown source in the block directly *}
{markdown}
# header {#header-id}

This is some markdown.
{/markdown}

So why separate the markdown instead of just injecting it directly into the smarty views or just writing it in HTML? Well, part of the requirements of the project was that any documentation written for the website should be able to be easily converted to a PDF. So how do we do that? After some research on what tools would be easy for others to use as well, it ended up being a simple process; in short:

1. Load markdown files, parse with Markdown Extra, converting to HTML.

2. Take the concatenated HTML and pipe the output into HTMLDOC.

HTMLDOC is an open-source command-line tool that converts HTML documents into PDFs. I decided to use this tool to automate the build of the PDF documentation. So for web viewing, Markdown is converted to HTML and cached on subsequent requests as HTML via Smarty, and for download, the PDF docs are created using the same Markdown; a nice separation of concerns, I thought.

Here's a section of the build-docs PHP script that is run during the ANT build for each site. Prior to this section, the file looks in the documentation and loads specified markdown files and parses them with Markdown extra and loads the concatenated results into a temporary HTML file. This file is then piped into HTMLDOC:

$command = "htmldoc ";
$command .= "--book "; // generate in book format with TOC
$command .= "--links "; // link up hyperlinks
$command .= "--title "; // include a title page
$command .= "--toctitle " . escapeshellarg($toc_title) . " "; // TOC title
$command .= "--linkstyle "underline" "; // what to do with hyperlinks
if ($title_image)
{
    $command .= "--titleimage " . escapeshellarg($title_image) . " ";
}
$command .= "--footer h./ ";
$command .= "--header .t. ";
//$command .= "--bodyfont helvetica ";
$command .= "-t pdf14 ";
$command .= "-f {$output_file} {$temp_file}";

exec($command, $result, $return);

We execute the htmldoc command via PHP and the documentation PDF is generated. The variables allow us to create a bootstrap file for each site's documentation to configure the output a little bit.

The final requirement for the documentation was the addition of tables of contents to any markdown file and documentation. The documentation writers wanted to be able to have a list of the <h1-6> tags linked up to the corresponding section on the page. I accomplished this by adding another parsing block to Markdown Extra. In short, this block uses Markdown Extra's existing list of parsed headers and wraps them in a <ul> list with anchor tag links. The regular expression for matching the "toc" is as follows:

"/{toc(?:|?([1-6])(?::([1-6]))?)?}/

Ain't regex lovely? This matches some of the following, replacing it with the rendered unordered list where it appears in the markdown document.

• {toc} renders the entire table of contents.
• {toc|3} would display <h1> through <h3> tags in the table of contents.
• {toc|2:5} would display <h2> through <h5> tags in the table of contents.

Anyway, that's some of the process stuff I've been plugging away on lately. I've sort of become obsessed with writing everything in Markdown now. If I need to send to another developer or client, I'll do something like the following:

alias markdown='/path/to/Markdown_1.0.1/Markdown.pl'
markdown my-doc-file.md | htmldoc --format pdf14 - > my-doc-file.pdf

(Obviously there'll be a bunch of params for htmldoc as seen in the PHP example above). Quite simply converting Markdown and piping to htmldoc. Of course, I don't get to use my fancy extended Markdown with tables of contents, but for day to day writing, it's ideal. Bam, lovely looking PDF.

Depending on the completion of these projects, I may release the updated Markdown Extra with Table of Contents and the related Markdown Smarty plugins. Check back later with me.