Let blocks in Minitest

I want to know the difference between a few constructs in Ruby, specifically in minitest-spec-rails, so today I’m going to write several similar tests exploring different pieces of the DSL that minitest-spec-rails exposes, and look at the differing behaviors that result.

I’ll start things off with an empty rails project, and a silly model:

rails g model Wizard name:string birthday:date level:integer

I don’t know why wizards care about birthdays, but I’m sure we’ll think of something. We need a method to test, so we’ll give the wizard something to do:

  def cast(spell)
    "I cast #{spell}!"
  end 

Now, what I’d like to do is write a test for this method, and then look at the Ruby bytecode and see what we can see. As a convenience, I want to make Ruby write the bytecode for the test file I care about to a file every time the test suite is run:

  def test_write_self_to_dsm
    code_str = File.read __FILE__
    asm = RubyVM::InstructionSequence.compile code_str
    assert File.write('test/models/dsm.txt', asm.disassemble), 'failed to write updated test file'
  end

And here are the tests that I care about:

 describe '#cast' do
    let(:default_spell) { 'magic missle' }

    it 'includes the input string in the return value' do
      spell = 'fireball'
      expected = true
      actual = wizard.cast(spell).include? spell
      assert_equal expected, actual, 'THE `IT` WITH THE LOCAL'
    end

    it 'is the same test as above but with a let block' do
      expected = true
      actual = wizard.cast(default_spell).include? default_spell
      assert_equal expected, actual, 'THE `IT` WITH THE LET BLOCK'    
    end
  end

The big, annoying uppercase messages in the assert args are to make it easy to figure out which block of bytecode corresponds to which it block.

And here’s the disassembled code for the first it block:

== disasm: #<ISeq:block (2 levels) in <class:WizardTest>@<compiled>>====
== catch table
| catch type: redo   st: 0002 ed: 0044 sp: 0000 cont: 0002
| catch type: next   st: 0002 ed: 0044 sp: 0000 cont: 0044
|------------------------------------------------------------------------
local table (size: 4, argc: 0 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: -1])
[ 4] spell      [ 3] expected   [ 2] actual     
0000 trace            256                                             (  22)
0002 trace            1                                               (  23)
0004 putstring        "fireball"
0006 setlocal_OP__WC__0 4
0008 trace            1                                               (  24)
0010 putobject        true
0012 setlocal_OP__WC__0 3
0014 trace            1                                               (  25)
0016 putself          
0017 opt_send_without_block <callinfo!mid:wizard, argc:0, FCALL|VCALL|ARGS_SIMPLE>, <callcache>
0020 getlocal_OP__WC__0 4
0022 opt_send_without_block <callinfo!mid:cast, argc:1, ARGS_SIMPLE>, <callcache>
0025 getlocal_OP__WC__0 4
0027 opt_send_without_block <callinfo!mid:include?, argc:1, ARGS_SIMPLE>, <callcache>
0030 setlocal_OP__WC__0 2
0032 trace            1                                               (  26)
0034 putself          
0035 getlocal_OP__WC__0 3
0037 getlocal_OP__WC__0 2
0039 putstring        "THE `IT` WITH THE LOCAL"
0041 opt_send_without_block <callinfo!mid:assert_equal, argc:3, FCALL|ARGS_SIMPLE>, <callcache>
0044 trace            512                                             (  27)
0046 leave                                                            (  26)

You can see on entry 0004 that the Ruby interprerter pushes the string literal 'fireball' onto the stack, then calls setlocal, which pops the stack and then assigns the object to the local variable table. The opt_send_without_block are method calls, and the getlocals after them get the local variables we set above.

Now let’s take a look at the it block that refers to the local set with let:

== disasm: #<ISeq:block (2 levels) in <class:WizardTest>@<compiled>>====
== catch table
| catch type: redo   st: 0002 ed: 0042 sp: 0000 cont: 0002
| catch type: next   st: 0002 ed: 0042 sp: 0000 cont: 0042
|------------------------------------------------------------------------
local table (size: 3, argc: 0 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: -1])
[ 3] expected   [ 2] actual     
0000 trace            256                                             (  29)
0002 trace            1                                               (  30)
0004 putobject        true
0006 setlocal_OP__WC__0 3
0008 trace            1                                               (  31)
0010 putself          
0011 opt_send_without_block <callinfo!mid:wizard, argc:0, FCALL|VCALL|ARGS_SIMPLE>, <callcache>
0014 putself          
0015 opt_send_without_block <callinfo!mid:default_spell, argc:0, FCALL|VCALL|ARGS_SIMPLE>, <callcache>
0018 opt_send_without_block <callinfo!mid:cast, argc:1, ARGS_SIMPLE>, <callcache>
0021 putself          
0022 opt_send_without_block <callinfo!mid:default_spell, argc:0, FCALL|VCALL|ARGS_SIMPLE>, <callcache>
0025 opt_send_without_block <callinfo!mid:include?, argc:1, ARGS_SIMPLE>, <callcache>
0028 setlocal_OP__WC__0 2
0030 trace            1                                               (  32)
0032 putself          
0033 getlocal_OP__WC__0 3
0035 getlocal_OP__WC__0 2
0037 putstring        "THE `IT` WITH THE LET BLOCK"
0039 opt_send_without_block <callinfo!mid:assert_equal, argc:3, FCALL|ARGS_SIMPLE>, <callcache>
0042 trace            512                                             (  33)
0044 leave                                                            (  32)

Here we can see in 0022 a method call: opt_send_without_block <callinfo!mid:default_spell. It looks like let blocks in minitest-spec-rails define methods. I did a little digging, and it turns out that let is part of minitest itself:

The code below is from: https://github.com/seattlerb/minitest/blob/f95ef007ec64d956d88fedc37fd2ed7e106b777e/lib/minitest/spec.rb#L236

    def let name, &block
      name = name.to_s
      pre, post = "let '#{name}' cannot ", ". Please use another name."
      methods = Minitest::Spec.instance_methods.map(&:to_s) - %w[subject]
      raise ArgumentError, "#{pre}begin with 'test'#{post}" if
        name =~ /\Atest/
      raise ArgumentError, "#{pre}override a method in Minitest::Spec#{post}" if
        methods.include? name

      define_method name do
        @_memoized ||= {}
        @_memoized.fetch(name) { |k| @_memoized[k] = instance_eval(&block) }
      end
    end

My best guess at why minitest is defining methods is that it can’t use instance variables or tests would become order-dependent – a test getting an instance variable would see changes made to that instance variable by previous tests, so tests would interfere with each other, and it doesn’t want to rely on local variables exclusively, because then tests would get repetitive if every local variable was dependent on the tests.

On the other hand, if every tests gets these fake instance variables as the result of a method call, it can mutate the return value as much as it pleases without messing up the other tests.

Next time, we’ll look more at how minitest and minitests-spec-rails use define_method calls to build up a DSL that lets us write more readable tests, and specifically I want to understand how the @_memoized cache of return values above is immune to being mutated.

Till next time, happy learning!

-Will