rspec - how to add more info to the Failure/Error where it occurs - ruby

I want to output a bunch of information from the test if it fails.
When i puts out the information however it appears before the Failures: section of the rspec output as opposed to where the specific spec failure info is (line number etc)
Is there a way in rspec for the spec to show the info in the failure itself rather than separately?
I thought mybe an around hook, but...
WARNING: around hooks do not share state with the example the way
before and after hooks do. This means that you cannot share instance
variables between around hooks and examples.```

You can use a lambda in your test:
expect(page).to have_text("Doesn't exist"), lambda { "This failed for all sorts of reasons, let me list them out here: #{detailed info}." }
Will give you output like:
Failures:
1) Blah blah blah
Failure/Error: expect(page).to have_text("Doesn't exist"), lambda { "This failed for all sorts of reasons, let me list them out here: nil." }
This failed for all sorts of reasons, let me list them.
# ./spec/features/search_results_spec.rb:19:in `block (2 levels) in <top (required)>'
It can be a bit tricky if you have code like expect(x).to eq y.count as just tacking on the lambda gives 2 params given but 0..1 expected. to get around this use formats like
expect(x).to (eq y.count), lambda { "message" }

Related

Unable to detrmine if a test failed in an after hook

I try to do some extra stuff when a test fails.
This is the failing test:
it 'fails' do
expect(1213).to eq('123456')
end
Following code is added to the spec_helper:
RSpec.configure do |config|
config.after(:each) do |example|
if example.exception
puts 'Do something'
end
end
Results in following output:
expected: "123456"
got: "1213"
(compared using ==)
1 example, 1 failure, 0 passed
Finished in 2 seconds
The exception on example stays nil, i do not understand wy. Is there another way to achieve extra code after a failing test?
Here is the answer (and an idea of the solution too) https://github.com/rspec/rspec-core/issues/2011#issuecomment-114669886:
Anyhow, the status is not set until AFTER the after hooks run. That is intentional because the after hook is itself part of the example, and if an exception occurs in the after hook we'll set the status of the example to :failed. after hooks are great for cleanup/teardown logic but isn't intended for observing the status of examples. Instead, I recommend you use a formatter for that...
The code should work perfectly, you should see "Do something" right before the "F" get printed on STDOUT, something like this:
Randomized with seed 3598
Do something
F
Failures:
1) Blah#blah fails
Failure/Error: expect(1213).to eq('123456')
expected: "123456"
got: 1213
(compared using ==)
# ./spec/index_spec.rb:9:in `block (2 levels) in <top (required)>'
Finished in 0.01617 seconds (files took 0.09568 seconds to load)
1 example, 1 failure, 0 passed
Also note which rspec version you are using, your test report looks different from mine. Mine is on v3.8

“Ambiguous match, found 2 elements matching visible link” issue

I've looked through a few posts with the same issue, but still feel like mine is a bit different.
viewing_categories_spec.rb
require 'rails_helper'
RSpec.feature 'Users can view categories' do
scenario 'with the category details' do
category = FactoryBot.create(:category, name: 'Real Estate')
visit '/categories'
click_link('Real Estate')
expect(page.current_url).to eq category_url(category)
end
end
category_factory.rb
FactoryBot.define do
factory :category do
name {"Computers"}
end
end
when I run rspec, I'm getting an error:
Failures:
1) Users can view categories with the category details
Failure/Error: click_link('Real Estate')
Capybara::Ambiguous:
Ambiguous match, found 2 elements matching visible link "Real Estate"
# ./spec/features/viewing_categories_spec.rb:8:in `block (2 levels) in <main>'
Then I've modified spec by adding match: :first:
require 'rails_helper'
RSpec.feature 'Users can view categories' do
scenario 'with the category details' do
category = FactoryBot.create(:category, name: 'Real Estate')
visit '/categories'
click_link('Real Estate', match: :first)
expect(page.current_url).to eq category_url(category)
end
end
This time I got error:
Failures:
1) Users can view categories with the category details
Failure/Error: expect(page.current_url).to eq category_url(category)
expected: "http://www.example.com/categories/265"
got: "http://www.example.com/categories/17"
(compared using ==)
# ./spec/features/viewing_categories_spec.rb:9:in `block (2 levels) in <main>'
I noticed that sometimes, I'm not seeing the error and sometimes it shown up.
The only thing I see always is "http://www.example.com/categories/17".
This part remains same always when I run rspec command.
The full source code is here https://github.com/tenzan/kaganat
The fact that the "http://www.example.com/categories/17" url is constant and that Capybara is seeing two "Real Estate" links on the page when your test appears to only create one leads me to believe that you have some old data left in your test database. By opting to use match: :first you've just covered up the fact that you have more records existing than you expect and that error should have been your first clue (along with just looking at a screenshot of the test running). Something like
rails db:reset RAILS_ENV=test
will clear out your test database and ensure you don't have old data hanging around. You'll also want to go back to your original click_link('Real Estate') without the :match setting. Additionally, if you want stable tests, you should almost never by using the standard RSpec matchers ('eq', etc) with Capybara returned objects since page load/behavior is an asynchronous thing. Instead you should should the matchers provided by Capybara. In your current example that means instead of writing expect(page.current_url).to eq category_url(category) you should be writing expect(page).to have_current_path(category_url(category))

Aruba not working with RSpec 3?

I'm trying to test a ruby command line script but am having troubles running tests with Cucumber-less Aruba in RSpec 3.
Some weird errors, some obvious ones.
e.g. weird:
1) test does something
Failure/Error: run_simple("cli_test.rb -h", true)
ArgumentError:
wrong number of arguments (2 for 1)
# .../.gem/ruby/2.2.0/gems/aruba-0.6.2/lib/aruba/api.rb:632:in `assert_exit_status'
# .../.gem/ruby/2.2.0/gems/aruba-0.6.2/lib/aruba/api.rb:750:in `run_simple'
# ./spec/cli_test_spec.rb:17:in `block (2 levels) in <top (required)>'
Looking in the code, I can't even see what's triggering this. Trying to use Pry or Pry-byebug to check out the erroneous 2 args isn't working on it either (a whole bunch of other errors).
Then, e.g. obvious:
1) test does something
Failure/Error: check_file_presence(["bin/cli_test.rb"], true)
only the `receive` or `receive_messages` matchers are supported with
`expect(...).to`, but you have provided:
#<RSpec::Matchers::BuiltIn::BePredicate:0x007fb36c88d6b0>
# ... error lines ...
& here, the errors are obviously correct, Aruba is using Rspec 2 syntax.
So I added
config.expect_with :rspec do |c|
c.syntax = [:should, :expect]
end
to my rspec config, but still not working.
Any ideas? Tips? Examples of this working anywhere?
Thanks in advance.
There are some conflicts using the examples for RSpec 2, specially if you are requiring aruba\api and messing up with the include/extend lines many of them have.
This is the smallest example to get Aruba working. You might want to fine tune your configuration with the PATH env and things like that
require 'aruba/rspec'
describe "Ruba example", :type => :aruba do
it 'shows root content' do
run 'ls /'
expect(all_stdout).to include('tmp')
expect(all_stdout).to include('var')
expect(all_stdout).to include('usr')
end
end

Why is my test double not expecting the command I allowed?

I have some code which makes shellout calls to the Linux OS, which will run distro-specific commands. I'm trying to ensure the tests can be run on any system, so am using a test double for the Mixlib::ShellOut call. Here's a simplified version that replicates my issue:
require 'mixlib/shellout'
class SelinuxCommand
def run
runner = Mixlib::ShellOut.new('getenforce')
runner.run_command
end
end
My test stubs Mixlib:ShellOut.new returning a test double, and then says that :run_command should return the string 'Enforcing':
require 'rspec'
require_relative 'selinuxcommand'
describe SelinuxCommand do
it 'gets the Selinux enforcing level' do
command = SelinuxCommand.new
Mixlib::ShellOut.stub(:new).and_return(double)
allow(double).to receive(:run_command).and_return('Enforcing')
expect command.run.to eql 'Enforcing'
end
end
However, when I run the test I see:
$ rspec -fd selinuxcommand_spec.rb
SelinuxCommand gets the Selinux enforcing level (FAILED - 1)
Failures:
1) SelinuxCommand gets the Selinux enforcing level
Failure/Error: expect command.run.to eql 'Enforcing'
Double received unexpected message :run_command with (no args)
# ./selinuxcommand.rb:5:in `run'
# ./selinuxcommand_spec.rb:9:in `block (2 levels) in <top (required)>'
Finished in 0.00197 seconds 1 example, 1 failure
Failed examples:
rspec ./selinuxcommand_spec.rb:5 # SelinuxCommand gets the Selinux enforcing level
I don't understand why the double doesn't expect :run_command when I explicitly set it up to expect that. What have I missed?
It is just because every time you call double you get a different object, so the object allowed to receive the run_command method is not the same object returned by the stubbed new. You can fix it like this:
it 'Gets the Selinux enforcing level' do
runner = double
Mixlib::ShellOut.stub(:new).and_return(runner)
expect(runner).to receive(:run_command).and_return('Enforcing')
expect(subject.run).to eq('Enforcing')
end
Can't check now but it seems to me that you need stub :initialize method - not :new.
Try this variant:
Mixlib::ShellOut.stub(:initialize).and_return(double)

How to write a particular rspec test? Advice needed …

I'm new to rspec and I'm trying understand how to write some particular tests.
Background:
I'm testing a Sinatra app
That app connects to several databases
I've defined the usernames/passwords for these databases as environment variables and I reference these variables in the code.
I have working test, so I know how to test for the existence of the environment variables.
Here's how I did it:
27 it "should know the username for each SUBDOMAIN in the list" do
28 #dm.domains.each do |dom|
29 ENV['C_USERNAME_' + dom].nil?.should eq(false)
30 end
31 end
As I said, the test works. The only problem is, when it fails, its unclear which dom doesn't have the environment variable set. The result of the failed test looks the following:
dhaskew#Air1:~/code/projects/rules$ rake spec
/Users/dhaskew/.rvm/rubies/ruby-1.9.3-p194/bin/ruby -S rspec ./spec/c_spec.rb ./spec/main_spec.rb ./spec/routes_spec.rb
..F*.....*
Pending:
My Site Domain Manager should know the password for each SUBDOMAIN in the list
# Not yet implemented
# ./spec/c_spec.rb:33
My Site GET '/rules/SUBDOMAINS' should test all subdomains
# Not yet implemented
# ./spec/routes_spec.rb:28
Failures:
1) My Site Domain Manager should know the username for each SUBDOMAIN in the list
Failure/Error: ENV['C_USERNAME_' + dom].nil?.should eq(false)
expected: false
got: true
(compared using ==)
# ./spec/c_spec.rb:29:in `block (4 levels) in <top (required)>'
# ./spec/c_spec.rb:28:in `each'
# ./spec/c_spec.rb:28:in `block (3 levels) in <top (required)>'
Finished in 0.05095 seconds
10 examples, 1 failure, 2 pending
Failed examples:
rspec ./spec/c_spec.rb:27 # My Site Domain Manager should know the username for each SUBDOMAIN in the list
rake aborted!
Question: How do I write this test(s) better so that I can see which environment variable isn't defined.
Thanks.
One easy solution is this: Generate the it tests dynamically. In general, strive for one test per scenario/it block. In this case:
#dm.domains.each do |dom|
it "should know the username for #{dom} in the list" do
ENV['C_USERNAME_' + dom].should_not be_nil
end
end
Update
Here's a more complete example. You need to declare #dm outside of a before block for it to be available.
class DomainManager
attr_reader :domains
def initialize(domains)
#domains = domains
end
end
describe DomainManager do
#dm = DomainManager.new( %w(example.com example.org))
#dm.domains.each do |dom|
it "should know the username for #{dom} in the list" do
ENV['C_USERNAME_' + dom].should_not be_nil
end
end
end

Resources