Hardest Question on the Ruby Object Model Quiz

Introduction

A few months ago we published a ruby object model quiz. This was a 10-question quiz, allowing you to self-test your knowledge of this fundamental component of the ruby language. It touched on concepts such as modules, inheritance, the ancestor chain, singleton classes etc.

Spoiler alert: If you haven't tried the quiz yet you might want to give it a go before reading the rest of this article!

Collecting the data from all respondants we can reflect on how people fared, see which questions posed the greatest challenge and carry out a retrospective, of sorts.

Breakdown by score

At the time of writing quiz had a total of 615 attempts registered, with an average score of 34%. Multiple attempts may have been registered by a single user, but I don't believe this fact will have much impact on our conclusions. The graph below shows the distribution of respondants' scores:

The average of 34% was low, but the questions posed were deliberately difficult and delved into the the inner workings of the Object ancestor chain; these are details which will often remain quite hidden when completing regular development tasks in the ruby language.

The time limit on each question was undoubtedly another significant factor in the scoring. Each question was allocated a time limit of 45 seconds, with the last two questions granted a little more time (60 secs). This really doesn't leave the quiz-taker much time to think through the response, and undoubtedly this has led to some rushed incorrect answers. In general, I think the time limit is a good thing; it forces the quiz-taker to push through to the end of the quiz in quick step and puts a hard-limit on the time cost to the quiz-taker. Importantly, the time limit on each questions makes it difficult for the quiz taker to copy-and-paste code into the console to check the output, which I feel would be contrary to the point. Notwithstanding, in retrospect I think the time limit could have been relaxed pretty significantly to allow people a fairer opportunity.

The modal score bracket was 30-39, with scores above 70% being very rare. So, if you are one of the souls who reached these upper echelons, kudos!

Breakdown by question

We now look at how users performed against the individual questions. You can click on the individual bars to see a screenshot of the question in each case.

The Hardest Question

We can see that the question that posed most problems was question 6, with only 5.16% of respondants getting this question correct. This question explores the methods on the Object class, and specifically we ask which of these methods come from BasicObject:

The BasicObject class is at the top of the ancestor chain. The Object class inherits this base class, but it also includes the Kernel module. So the question we are asking is, which of the listed methods does Object inherit from BasicObject, and which does it pull in from the Kernel module? We can quickly discover the answer using irb console:

    
>> Object.ancestors
=> [Object, Kernel, BasicObject]
>> BasicObject.instance_methods
=> [:!, :equal?, :__send__, :__id__, :==, :!=, :instance_eval, :instance_exec]
    
  

So we only get a mere 8 methods from the BasicObject class. They are pretty foundational, but not the sort of methods we use on a daily basis; the __id__, == and != deal with object identity, but the more familiar object_id is actually introduced by the Kernel module.

Similarly, the (presumable) building blocks for method invocations can be seen on the BasicObject interface through __send__, instance_eval and instance_exec; but conveniences like :send, :public_send, :tap etc. are coming from the Kernel module.

So in relation to question 6, of the listed methods, the only ones which are inherited from BasicObject are #equal? and #!=. Tough question and it is not surprising that so many respondants failed to answer it correctly!

As a basic sanity test of our understaning, let's look at the number of instance methods exposed by each of the components in the ancestor chain:

    
puts "BasicObject"
puts "All instance methods: #{BasicObject.instance_methods(true).size}"
puts "Own instance methods: #{BasicObject.instance_methods(false).size}"

puts "Kernel"
puts "All instance methods: #{Kernel.instance_methods(true).size}"
puts "Own instance methods: #{Kernel.instance_methods(false).size}"

puts "Object"
puts "All instance methods: #{Object.instance_methods(true).size}"
puts "Own instance methods: #{Object.instance_methods(false).size}"
    
  

Running the previous script will output the following:

    
BasicObject
All instance methods: 8
Own instance methods: 8
Kernel
All instance methods: 43
Own instance methods: 43
Object
All instance methods: 51
Own instance methods: 0
    
  

We see a total of 51 instance methods exposed on the Object interface, 8 of these coming from the BasicObject class with the remaining 43 being introduced by the Kernel module.

Summary

We have presented a short analysis of the results registered against our recently published Ruby Object Model quiz. The low average score suggests that the questions were a bit too difficult, and we propose that the subject matter of the questions was often outside the everyday experience of most ruby developers. We have also hypothesised that scores would have been better had the question time-limit not been so agressive.

In terms of performance against specific questions, the poorest performance was registered against question 6. This related to specific methods which exist on the BasicObject class, versus the Kernel module. For most rubyists this is an implementation detail which they are unlikely to worry about in their day-to-day development. As such, this question, along with many of the others in this quiz, are probably best regarded as interesting implementation trivia, rather than any sort of measure of competence.