Operator Precedence Riddles in Ruby

Introduction

As a relatively experienced developer I am frequently reminded of my own stupidity. If not stupidity, then certainly my propensity to be confounded by questions of basic logic. One recent episode involved sinking waaaay too much time into trying to resolve a test failure which boiled down to a simple question of operator precedence. Operator precedence is the set of rules used by a programming language (i.e. its interpreter or compiler) to decide the order in which it should apply operations, when evaluating an expression that contains multiple of such operations. The typical example cited is a mathematical expression such as

    
2+10*3 # Evaluates to 32
    
  

This expression evaluates to 32 because the multiplication operator (*) has a higher precdence than the addition operator (+). This particular ordering applies in mathematics and in any programming language I have ever come across. We can use brackets to override the natural ordering in expressions. For example, if we wanted the addition to happen first we could express this as follows:

    
(2+10)*3 # Evaluates to 36
    
  

With that quick recap out of the way, let's look at the essence of the problem that led to my consternation, and perhaps it may help you avoid similar frustrations.

The problem

Let's start with a simple Token class. We instantiate the class with a string value that is maintained in the internal state of the instance (@token) along with an @error variable. This @error variable is simply intended to record any errors that are encountered during the lifetime of the instance.

We have a downcase! method that will downcase the token and update the internal state, @token. If the token has not been set then we should update the @error variable and exit the method early. This simple class is listed in its entirity below:

    
class Token
  attr_reader :token, :error

  def initialize(token)
    @token = token
    @error = "none"
  end

  def downcase!
    (@error = "no_token" && return) unless token
    @token = token.downcase
  end
end
    
  

We can use this class as follows:

    
t = Token.new("BLAH")
t.downcase!
puts t.token     # Outputs 'blah'
puts t.errors     # Outputs 'none'
    
  

Inspecting the state of the instance after calling downcase! we can see that the @token variable is downcased and @error remains equal to its initial default value, 'none'. No shocks there.

If we now create an instance with a nil string:

    
t = Token.new(nil)
t.downcase!
puts t.token     # Outputs nothing
puts t.errors     # Outputs 'none' ??
    
  

In this case the @token variable remains nil but the @error instance variable has not been updated to reflect no_token. That wasn't what I was expecting, what has happened?

The explanation

Our problem is lurking in this line of the downcase! method:

    
(@error = "no_token" && return) unless token
    
  

We want to set the @error variable and return from the method when the token is not set but the assignment to @error does not seem to be happening. When you suspect an issue with your operator precedence there are two reasonable approaches you can adopt:

1. Refer to the documentation
2. Liberally add brackets until things work as you expect

Option 2 is probably less yawn-inducing, so I tend to go with that approach first. Indeed, my own proclavity for brackets can be witnessed in this simple example; notice how I use brackets to group the expression to be evaluated when the token is absent, (@error = "no_token" && return) . This use of brackets is completely redundant here due to the low precedence of the unless modifier. Nonetheless, I choose to leave the brackets in as they sometimes help the reader to visually parse the program logic.

I digress, back to our 2 options. In using brackets we are effectively defining our own custom precendence and, in so doing, we learn nothing about the inherent precedence in the language. But, given that we have arrived at this point owing to our lack of familiarity with this implicit precedence, it might be a good idea to try option 1 from time to time. Anyway … let's add some brackets.

After one or two permutations you should quickly realize that the following configuration:

    
((@error = "no_token") && return) unless token
    
  

will give us the result we desire:

    
t = Token.new(nil)
t.downcase!
puts t.token # Outputs nothing
puts t.errors # Outputs 'no_token' 
    
  

We needed the brackets to group the assignment (=) and separate from the && return. And, if we take a look at the operator precedence in the docs, we can see that, indeed, the = operator has a lower precedence than &&. Our original broken version was effectivly evaluating like this:

    
(@error = ("no_token" && return)) unless token
    
  

The right-hand-side of the assignment will evaluate first, i.e. ("no_token" && return), but the assignment to @error never completes because the return executes. This explains why the @error variable remained equal to 'none'.

In looking at the operator precendence table in the documentation you might have noticed that whilst && has a higher precendence than =, the and operator actually has a lower precedence. This means that, alternatively, we can fix our logic by replacing the operator && with operator and, which will again yield the desired result:

    
(@error = "no_token" and return) unless token
    
  

And just to completely flog this example to death we look back at our original implementation:

    
(@error = "no_token" && return) unless token
    
  

and note that we can introduce a whole new level of confusion for ourselves on account of the the short-circuit behaviour of the && operator. Suppose that, instead of setting @error to the value 'no_token' when the @token is absent, we choose to set @error to some falsey value in this case, e.g. nil:

    
(@error = nil && return) unless token
    
  

What happens in this case? We actually get a NoMethodError on the downcase method. The nil value causes a short-circuit in the && expression, @error gets assigned the nil value correctly, but the return statement is never reached and execution falls through to the next line in the method, which attempts to invoke downcase on the empty @token.

Summary

Operator precedence is the set of rules which a programming language uses to decide in what order an expression should be evaluated when it contains multiple operators. We have looked at a simple example demonstrating how we can easily introduce bugs into our code without a solid grasp of the precendence of different operators, specifically the example looked at the interplay of the and, && and = operators in ruby. To help resolve such confusion we can employ brackets to group sub-expressions, clarify our intentions and provide visual guidance for readers of our code. Brackets or not, we should still aim to have a solid grasp of the natural precedence of operators, lest we get caught out.

References

1. Ruby docs for operator precedence