Marko Klemetti

mrako

Folder Excludes for Modern JavaScript in Sublime Text

Are you developing using Grunt, Bower and node / npm?

You might want to exclude the default package folders from your Sublime workspace. This way the precompiled or massive library files will not be shown in the navigation or search of your own development.

Edit your user settings by either going to Sublime Text > Preferences > Settings - User or pressing Cmd-, (in OSX).

Add the folder exclusions to the settings:

"folder_exclude_patterns": [".sass-cache", ".tmp", "node_modules", "bower_components"]

Add Inverse_of to Upgrade to Rails 3.2.11 (to Fix the Security Hole)

As you hopefully already know, there is a severe security issue in Rails versions before 3.2.11, 3.1.10, 3.0.19 and 2.3.15.

If you are running Rails 3.2.8 or below, and have not yet added inverse_of to your ActiverRecord::Associations, you might be in for a big bunch of changes before the update.

Autoupdating Your Models

I created a small script to update your models. It’s not that pretty, but does what is expected: updates has_many and belongs_to associations with the correct inverse_of clause.

add_inverse_of.rb

#!/usr/bin/env ruby
require 'active_support/inflector'

Dir.glob("#{Dir.pwd}/*.rb") do |file|
  basename = File.basename(file, ".rb")

  lines = IO.readlines(file).map do |line|
    if line.match(/(belongs_to|has_many (((?!through)(?!inverse_of).)*)$)/)
      line.sub(/has_many ([^\n]*)/){|m| "has_many #{$1}, inverse_of: :#{basename}"}.sub(/belongs_to ([^\n]*)/){|m| "belongs_to #{$1}, inverse_of: :#{basename.pluralize}"}
    else
      line
    end
  end

  File.open(file, 'w') do |file|
    file.puts lines
  end
end

It’s very simple to use. If you have Rails installed in your RVM environment, just run the following in the root of your Rails App:

$ cd app/models
$ ruby <location of the script>/add_inverse_of.rb

If you have a clean Ruby environment, you might need to run $ bundle exec ruby add_inverse_of.rb

Fixing other Issues

You might not have created all of your model associations as bi-directional in the first place, as the earlier Rails versions didn’t force it. This is a very good time to fix the consistency of your app and check your factories and specs (or any other combination of fixtures and tests) verify your application still works as expected.

How to Write Good User Stories

User stories are the core for Agile and User-centered Product Development. As the focus of software development has transferred from making the product work to making the product serve the users’ needs, user stories have become an important tool for communication between the users and the developers.

This post introduces the concept of user stories and the best practices for writing them.

What User Story?

User Story describes an independent requirement for a system from the user’s point of view. A good user story briefly explains what, to whom and why.

Usually users stories are written in three parts. The first part explains who in form of As a <user role>. The second part shortly introduces the actual requirement: I want to <description of the goal>. The third part is the motivation: Why should this requirement be implemented or fulfilled. So that <a motivation>.

An example of a user story for writing a tweet:

As a socially active person
I want to update my status
So that I can share my thoughts with others

User Stories are often written on index cards or sticky notes, but there are also good requirement management applications from light-weight like Storywall to heavy-weight like Atlassian Jira. My current favorite is Pivotal Tracker.

Best Practices for writing User Stories

  • User stories should catch real benefit
  • User stories should generate giscussion
  • User stories can be estimated
  • User stories can be decomposed

User Stories Should Catch Real Benefit

A good user story very clearly states why it should be implemented. The user story has a value that can be compared with the other user stories, and that helps deciding if the user story is valuable enough for implementation.

User Stories Should Generate Discussion

The purpose of a user story is to generate discussion. As user stories are short and condensed, the essence behind a good story lies in the tacit knowledge created together with the developers and the users.

User Stories Can Be Estimated

Reliably estimating the work required to implement a user story is often very hard. I recommend you to create a relative scale and estimate the size of your user stories compared to the other user stories.

It’s often a good idea to give points for each user story and thus value the workload. I like using the powers of two (1, 2, 4, 8), but there are many other techniques as well. Pick one and try it out!

User Stories Can Be Decomposed

tdd

Good user story can and should be split into tasks, smaller yet deliverable parts of the original story. The tasks make the implementation work easier to understand, and helps following the progress more accurately. Tasks also guide the work on a team level, especially if more than one person is working on the same user story at the same time.

Creating and Overriding Rvmrc

rvmrc is a file which together with RVM makes sure you are always running the correct ruby version and gems for your application. You can set a project-specific environment with a file called .rvmrc by running the following command in your project root:

$ rvm --rvmrc --create jruby-1.7.0@mygemset

Now as you enter your project-folder, RVM automatically selects jruby-1.7.0 and uses the rubygems installed in the gemset mygemset.

Simpler rvmrc file

Since RVM creates a slightly bloated .rvmrc, you can also create the same environment file by manually writing it:

$ echo "rvm use jruby-1.7.0@mygemset" > .rvmrc

It is much cleaner and simpler to understand.

Overriding rvmrc

You probably never have to override your rvmrc. However I have an application which is using JRuby and I deploy it using Capistrano. Unfortunately JRuby doesn’t properly support Capistrano and I have to deploy the application by running:

$ rvm ruby-1.9.3-p327 do cap deploy

It selects another ruby version (ruby 1.9.3) for the one command only. And if you want, you can naturally create a rake task or a shell alias for running the command for you.

Highlight CSS Colors in Sublime

I usually create my layouts in Photoshop, but often when I’m implementing the layout (in Sublime) it would help me to immediately know what color the HEX values actually are.

I tried using Sublime ColorPicker, but it starts a separate application and I have to close the Colorpicker Window before continuing. Thus it was blocking my workflow.

Luckily I found the ColorHighlighter plugin, which highlights the HEX value with its respective color.

Installation

  1. Install the Sublime Package Control
  2. Press Shift-Cmd-P to open the Command Palette
  3. Type “insta” and select Package Control: Install Package
  4. Find and install Color Highlighter

Move your cursor on the HEX value and it will be highlighted with the color it represents.

Pursuing High-Speed Development: Sublime, RVM and RubyTest

This is the fifth episode of the series Pursuing High-Speed Development. It took me a while to solve how to best run RSpec (and other tests) while using RVM in Sublime. This text should let you start testing your Ruby projects in Sublime with the least effort.

My requirements for the functionality

  • One shortcut for running all tests in the current file
  • One shortcut for running the current test (where the cursor is)
  • Using the same independent environment as in my shell
  • Test results visible in Sublime (not in external terminal screen)

Installing RVM

RVM is easily the best tool to keep your ruby versions and gemsets separated. To install RVM, just run

$ curl -L https://get.rvm.io | bash -s stable --ruby

Now you can use rvm-auto-ruby, a tool that sets up your shell with the correct ruby version and gemset:

$ ~/.rvm/bin/rvm-auto-ruby -S <your ruby command>

Install RubyTest in Sublime

If you have not yet installed Package Control for Sublime, do it now.

With Package Control you can install new packages by pressing Shift-Command-P and selecting Package Control: Install Package (or just type “ins”).

Once the available packages have been loaded, find RubyTest and press enter.

Configure RubyTest

Once you have installed RubyTest, you can change the configurations in Sublime Test 2 -> Preferences -> Package Settings -> RubyTest. Pick Settings - User and add the following configuration for RSpec:

{
  "run_rspec_command": "~/.rvm/bin/rvm-auto-ruby -S rspec {relative_path}",
  "run_single_rspec_command": "~/.rvm/bin/rvm-auto-ruby -S rspec {relative_path} -l{line_number}"
}

If you are using Test::Unit or Cucumber, pick the configurations from Settings - Default change the configurations accordingly.

Additionally you can change the default keyboard shortcuts in Key Bindings - User:

[
  { "keys": ["super+shift+t"], "command": "run_single_ruby_test" }, // single test
  { "keys": ["super+shift+r"], "command": "run_all_ruby_test" }, // test file
]

Always remember to make your Sublime settings in User, not in Default. Otherwise your settings might get overridden when the package is updated.

Runing tests in Sublime

Press Shift-Command-R and voila:

..

Finished in 0.00056 seconds
2 examples, 0 failures
[Finished in 1.5s]

Python: Creating Your Project Structure

I love using Python for creating command-line applications which require just a bit more logic than writing a bash-script.

Python has excellent libraries for parsing command-line arguments and running other shell commands, while at the same time you can take advantage of the powerful object-oriented language. Additionally you can verify and document your application with the Python’s unit testing framework.

I’ve created an Example Application for this text, and you can find the source code in github.com/mrako/python-example-project.

Structuring your Application

In my experience the best folder structure for a Python project is to put your executables in a bin folder and your project into a your project name folder (I will use name project in this text). This way you can keep your core functionalities separated and reusable. It is also a standard for other applications.

In project folder you should have main.py as the main access point to your application. Your functionalities should then go to lib and your unitests to tests. This way the skeleton of your project should look like:

bin
  project
project
  main.py
  + lib
  + tests

Now your application can be executed by running:

$ bin/project <parameters>

Separating Parameters, Shell Commands and Functionalities

As in all object-oriented programming you should aim to separate your concerns. It’s often forgotten in Python applications because reading the command-line parameters, handling options and running other shell commands is very easy.

Parsing the Command-Line Options

Create a class which defines and collects the command-line parameters. Python provides optparse, for which you can very easily define the options and behavior:

usage = 'bin/project'
parser = OptionParser(usage=usage)
parser.add_option('-x', '--example', default='example-value',
                  dest='example', help='An example option')

Now you have created a parser which reads a target value to the example variable by running bin/project -x <target value> or bin/project --example <target value>

Running other Shell Commands

If you want to create an application which depends on other shell commands you should separate the shell execution in its own class. This way your core functionalities can be easily re-used in other environments or applications, and you can more easily collect logs, errors and exceptions that come from external sources. I recommend you to use the Python subprocess for your external shell commands.

Create a classes for the process execution and the process exceptions (see process-class in the example project):

import subprocess

class Process:
  def execute(self, command):
    process = subprocess.Popen(shlex.split(command),
                               stdout=subprocess.PIPE,
                               stderr=subprocess.PIPE)
    out, err = process.communicate()
    if process.returncode:
      raise ProcessException(process.returncode)
    return out

class ProcessException(Exception):
  def __init__(self, value):
    self.parameter = value

  def __str__(self):
    return repr(self.parameter)

The Core Functionalities

In your project/lib/project.py you can now implement the core functionalities. Since this is an example I’ve only included receiving options(which have been separated from collecting the command-line arguments) and running the date command through your Process-class:

from process import Process

class Project:
    def __init__(self, options):
        self.options = options
        self.process = Process()

    def date(self):
        self._get_date()

    def _get_date(self):
        print self.process.execute("date")

Running your app

Call your project’s main.py from the bin/project executable:

#!/bin/bash

BINPATH=`dirname $0`
python "$BINPATH/../project/main.py" $@

Don’t forget to change the access rights for execution:

$ chmod 755 bin/project

In main.py you then collect the command-line options and run your application:

import sys

from lib import Project
from lib import Options

if __name__ == '__main__':
    options = Options()
    opts, args = options.parse(sys.argv[1:])
    v = Project(opts)

    print v.date()

Finally you are ready to run the application:

$ bin/project <arguments>

Testing (and Documenting)

As already written in Unit Testing in Python and Short Introduction to Unit Testing, I cannot emphasize the importance of testing enough. With unit tests you can guide the development, verify your functionalities and document the behavior of your application.

Add your tests in the project/tests folder (see tests-folder in the example project). I recommend you to use nose for running your tests.

Testing Command-Line Arguments

import unittest

from lib import Options

class TestCommandLineArguments(unittest.TestCase):
    def setUp(self):
        self.options = Options()

    def test_defaults_options_are_set(self):
        opts, args = self.options.parse()
        self.assertEquals(opts.example, 'example-value')

Running the Tests

$ nosetests
.....
----------------------------------------------------------------------
Ran 5 tests in 0.054s

OK

Unit Testing in Python

Many think unit testing is difficult, complicated or requires lots of boilerplate code. This is however not true. Consider the following example in Python (let’s name it unittest_example.py):

import unittest

class RomanNumerals:
  def convert(self, number):
      return "I"

class TestRomanNumeralsConverter(unittest.TestCase):
    def test_returns_I_for_1(self):
      self.assertEquals(RomanNumerals().convert(1), 'I')

if __name__ == '__main__':
  unittest.main()

I can hear some of you thinking “Cannot be true!” and “What about the dependencies, and installations?”, but believe or not this code works in almost every OSX and Linux distribution out of the box by running:

$ python unittest_example.py

.
----------------------------------------------------
Ran 1 test in 0.000s
OK

And by installing nose you can run all unit tests in your project with

$ nosetests

For more information and the rest of the assertion types, check out the official python unittest documentation

More about python projects, python installations and nosetests in Python: Creating Your Project Structure.

Pursuing High-Speed Development: Serene Testing

This is the fourth episode of the series Pursuing High-Speed Development.

If you have ever tried Unit Testing or even Test-driven Development, you already know it’s not going easy to maintain the same velocity and rhythm you’re used to. Often you will spend your valuable development time thinking “How on earth am I going to test this?” or “Breaking the dependency between these modules is killing me!” Instead you would rather hack together your features and mark your tasks “done”.

Well, truth is testing is not easy. But it’s not that hard either. With testing it’s just very easy to get lost spending your time on everything else than what’s important. The purpose of this post is to guide you through the common pitfalls of unit testing.

Think Simple

Simplicity is very rarely a weakness in software code. The only important thing is to meet your customers needs, and the simpler you can achieve it - the better.

Unit Tests help you achieve simplicity by forcing you to separate concerns. With unit testing you will almost automatically write smaller units and keep your code minimal. If you however end up spending time on anything other than that, you have not achieved simplicity. You will very probably stop writing tests and thus the benefits are lost.

Avoid spending time on complex test preparation or writing unnecessary boilerplate code to get your tests working.

Fail Early

To get the most out of unit testing, it is very important to execute your tests all the time. If you don’t, or can’t, run the tests after each change, it is very probable you don’t have enough motivation to write them hand-in-hand with the actual program code. And I don’t blame you - why write extra code if there is no value?

However tests create the body of your software. Your tests help you creating and refactoring better code, and good tests inform you immediately if you’ve made a mistake or a change that has unwanted side effects.

Make sure you can receive the value immediately.

Fail Fast

XKCD: Compiling

If your tests take time to run, you will quickly run into another major obstacle: waiting. Xkcd has expressed this problem the best so far (replace compiling with testing).

The purpose of unit tests is to write the verification so that the feedback could be received as quickly as possible. If you don’t limit the scope of your unit tests, and stub or mock your time-consuming interfaces, the total execution time will soon get out of hands. And then you will not run the tests anymore.

Navigate Quickly Between Code and Tests

It is natural to switch between your tests and the code all the time. If switching is complicated or takes time, you have once again created an unnecessary obstacle for smooth testing.

Many split the screen between tests and code, but personally my flow works better when switching between tabs. You should try out a few different practices and select the one you feel most comfortable with. For quicker switching, avoid at least:

  • Using mouse to switch between the code and the tests
  • Continuously searching for the tests or the code file
  • Writing all tests (or all code) first and only until then switching to the code (or tests)

Tips

  • RubyTest brings the easy way for developing Ruby in Sublime.
  • I’m using Shift-Ctrl-Down to navigate between the code and the tests.
  • Default switch for RubyTest is Command-., you can configure it from Sublime Text 2 -> Preferences -> Package Settings -> RubyTest

Auto-test

I usually start the unit test run with a keypress, but you might want to try auto-testing. Auto-test application follows changes in your project, and once you save it automatically runs the related tests.

I’ve found Guard to be the best tool so far. It is very easy to add into your project and almost as easily configured. I’ve added Guard to the Roman Numerals Example.

To use Guard, you need to make sure you have installed RVM (see Introduction) and Bundler. Then add Guard to a Gemfile in your project root:

group :development do
  gem 'guard'
end

Now you have installed Guard, and you can create your Guardfile. Guardfile for the example project is making Guard run RSpec when spec/*_spec.rb or any other *.rb file is updated:

guard "rspec", :version => 2 do
  watch(%r{^spec/.+_spec\.rb$})
  watch(%r{^(.+)\.rb$})   { |m| "spec/#{m[1]}_spec.rb" }
end

The How’s, Why’s and Why Not’s of Test-driven Development

This post is based on Unit Testing and introduces the concept of Test-driven Development.

Test-driven Development, or TDD, is a practice where tests guide the development flow. Instead of just jumping right into coding, the developer first writes a failing Unit Test, describing a single, separated functionality. Developer then writes just enough code to make the test pass, refactors the code structure and verifies all the tests pass. The developer then continues to write the next test. This is called the TDD cycle.

The motivation for TDD is to drive the development outside the code and help creating a better design. In TDD you automatically think test-first (see Unit Testing), which means you concentrate on what needs to be done and verify the functionality at the same time.

What Refactoring?

Martin Fowler has described Code Refactoring so well that I rather guide you there:

Red - Green - Refactor

tdd

The TDD cycle is often called Red - Green - Refactor to better describe the development flow:

  1. Red: Write a failing test.
  2. Green: Make the test pass with as little coding as possible.
  3. Refactor: Eliminate duplication and improve the design.

In TDD it’s very important to first make sure the new test, and only the new test, is failing after Red. From the failing test you know there is a functionality missing. After Green it is as important to make sure all tests pass to make sure your code is working as expected. You can then proceed to Refactor where you remove duplication and improve the overall design of your code. As you now know all tests pass for your existing functionality, Refactoring is safe and you can change the code structure without the fear of breaking something.

Once you are happy with the code, you start the TDD cycle again with a new test.

How about Behavior-driven Development?

Many argue the example below is actually Behavior-driven development, a practice derived from TDD. The difference is that in BDD the system requirements are more strongly connected to the test cases and thus the tests create a stronger behavioral description of the software module.

In my experience quality code should be anyway understood by others, and the tests should describe the functionalities of the software in an understandable way. Thus I don’t like making a separation between these two terms.

Critic against Test-driven Development

This is the hard part. I personally feel that Test-driven development is a very important practice and every developer should have TDD in their toolbox. However, TDD is a very strict practice and often testing every line of code might prove to be impossible. I wouldn’t want to feel like a loser for not creating e.g. unit tests for threaded backend processes or skipping UI tests for the sake of already having Acceptance Tests in place.

As I already wrote in Unit Testing, the developers should always think test first, because that is one of the most important benefits of TDD - guiding the development from outside the code. On the other hand strictly writing tests first might occasionally decelerate or hinder the development flow.

Finally: An example of the TDD Cycle

I’ll use the familiar Roman Numerals (see Unit Testing) as an example. The examples have been written in Ruby and RSpec, but don’t worry, you will be able to follow the idea even without knowing the syntax.

1. Creating the module and testing 1 converts to “I” (Red)

Let’s start with the first failing test by creating the file spec/roman_numeral_spec.rb:

require "roman_numeral"

describe "roman numerals" do
  subject { RomanNumeral }

  it("should convert 1 to 'I'") { subject.convert(1).should == "I" }
end

And run it to make sure it really fails:

$ rspec

... `require': cannot load such file -- roman_numeral (LoadError)

2. Make it pass (Green)

As we don’t have the module nor one single line of code it is natural that the test is not passing. Now let’s continue to implement the code. Create file lib/roman_numeral.rb:

class RomanNumeral
  def self.convert(arabic_number)
    return "I"
  end
end

And run the tests:

$ rspec
.

Finished in 0.00044 seconds
1 example, 0 failures

Perfect. Currently the code is not doing much, and there is not much to refactor, so we move on to the next cycle.

3. Roman numerals should return “II” for number 2 (Red)

Let’s add a new test to verify the converter returns “II” for number 2:

require "roman_numeral"

describe "roman numerals" do
  subject { RomanNumeral }

  it("should convert 1 to 'I'") { subject.convert(1).should == "I" }
  it("should convert 2 to 'II'") { subject.convert(2).should == "II" }
end

And run the tests to verify only the new test is failing:

$ rspec
.F

Failures:
  1) roman numerals should convert 2 to 'II'

4. Make it pass (Green)

Let’s write just enough code to make the new test pass:

class RomanNumeral
  def self.convert(arabic_number)
    if arabic_number == 1
      "I"
    else
      "II"
    end
  end
end

And run the tests:

$ rspec
..

Finished in 0.00055 seconds
2 examples, 0 failures

5. Removing duplication (Refactor)

As we have now verified our system works as expected we can refactor the code to remove duplication, clean up the structure and make changes towards the final solution.

class RomanNumeral
  ROOT_NUMERALS = {1 => "I", 2 => "II"}

  def self.convert(arabic_number)
    return ROOT_NUMERALS[arabic_number]
  end
end

You can now proceed to finalizing the solution and sending me a Github Pull Request. Happy coding!