Applitools Eyes Calabash SDK

⚠️ DEPRECATION NOTICE: The Calabash framework is no longer actively maintained. Microsoft discontinued their contributions after iOS 11 and Android 8 support, and Visual Studio App Center (which integrated with Calabash) is retiring on March 31, 2025. For mobile visual testing, we strongly recommend using the Applitools Eyes Appium SDK instead, which provides better support and active development for mobile applications.

The Applitools Eyes Calabash SDK brings visual testing to your Calabash mobile automation. Test native mobile apps on Android and iOS with AI-powered visual validations.

Overview
Installation
Quick Start
Key Components
Advanced Usage
Integration with Cucumber
Platform-Specific Features
- Android Specifics
- iOS Specifics
Best Practices
Common Pitfalls
FAQ

Overview

The Applitools Eyes Calabash SDK integrates visual testing with Calabash mobile automation framework. It enables you to capture and validate the visual appearance of your mobile applications on both Android and iOS platforms.

Key Features

Cross-Platform Support: Works with both Android and iOS applications
Cucumber Integration: Pre-defined step definitions for easy test writing
Scrollable Content: Automatic capture of scrollable views
Platform Detection: Automatically adapts to the target platform
Region-Based Testing: Focus on specific UI elements
RSpec Matchers: Custom matchers for visual validation
Debug Support: Save screenshots for troubleshooting

The SDK handles platform-specific complexities such as screenshot capturing, element detection, and scrollable content stitching, allowing you to focus on writing tests rather than dealing with implementation details.

Installation

Add the gem to your Gemfile:

gem 'eyes_calabash'

Or install it directly:

gem install eyes_calabash

Dependencies

The Eyes Calabash SDK has the following dependencies:

eyes_core - Core Applitools functionality
calabash-android or calabash-cucumber for iOS tests

Note: The SDK operates with minimal dependencies for better compatibility across different environments.

Quick Start

Here's a simple example of using Eyes Calabash in a Cucumber feature file:

Feature: Visual Testing with Calabash

  Scenario: Check the home screen
    Given I start the app
    And I use Eyes with app name "My App" and test name "Home Screen Test"
    Then I check the window with tag "Home Screen"
    And I close Eyes session

These steps are automatically provided by the SDK. Behind the scenes, they execute the following Ruby code:

require 'eyes_calabash'

# Initialize Eyes
eyes = Applitools::Calabash::Eyes.new
eyes.api_key = ENV['APPLITOOLS_API_KEY']

# Start the test
eyes.open(app_name: 'My App', test_name: 'Home Screen Test')

# Perform a visual validation
eyes.check('Home Screen', Applitools::Calabash::Target.new)

# Close the test
eyes.close

Key Components

Eyes Class

The Applitools::Calabash::Eyes class is the main entry point for the SDK. It provides methods for:

Starting test sessions with open
Capturing and comparing screenshots with check
Ending test sessions with close
Setting capture options like viewport size, match level, etc.

It automatically detects whether you're running on Android or iOS and adjusts its behavior accordingly.

Target Class

The Applitools::Calabash::Target class defines what to capture during a check operation:

# Check the entire screen
eyes.check('Home Screen', Applitools::Calabash::Target.new)

# Check with full page capturing (scrolling)
eyes.check('Scrollable Screen', Applitools::Calabash::Target.new.fully)

# Check a specific element
element = query("* marked:'username'").first
calabash_element = Applitools::Calabash::CalabashElement.new(element)
eyes.check('Username Field', Applitools::Calabash::Target.new.region(calabash_element))

Screenshot Management

The SDK provides platform-specific screenshot handling:

Android-specific screenshot capture
iOS-specific screenshot capture

These features handle capturing screenshots at the right resolution, accounting for device pixel ratios, and managing debug screenshot captures.

Environment Detection

The EnvironmentDetector class automatically determines whether your test is running on Android or iOS, ensuring the right platform-specific code is used.

Scrollable Content Handling

One of the most powerful features is the ability to capture scrollable content with the fully method:

# Capture a scrollable view by automatically scrolling and stitching
eyes.check('Full Page', Applitools::Calabash::Target.new.fully)

The SDK includes specialized algorithms for different scrollable containers:

Android ScrollView
iOS UITableView

Advanced Usage

Standardized Screenshot Handling

The SDK offers standardized screenshot handling:

# Initialize Eyes
eyes = Applitools::Calabash::Eyes.new

# Perform a check with optimized processing
eyes.check('Home Screen', Applitools::Calabash::Target.new)

Benefits include:

Cross-platform consistency
Platform-optimized image processing
Accurate region identification
Efficient execution

Note: The SDK maintains backward compatibility with existing tests.

Region-Based Testing

You can check specific regions of the screen by using the Calabash query mechanism:

# Find an element using Calabash's query and create a Calabash Element
element = query("* marked:'submit_button'").first
calabash_element = Applitools::Calabash::CalabashElement.new(element)

# Check just that element
eyes.check('Submit Button', Applitools::Calabash::Target.new.region(calabash_element))

Full Page Capture

For scrollable content, use the fully method on the Target class:

# This will automatically scroll and capture the entire content
eyes.check('Scrollable Content', Applitools::Calabash::Target.new.fully)

The SDK automatically detects common scrollable containers and uses the appropriate scrolling algorithm.

Ignoring Regions

To ignore specific regions during comparison:

# Find an element to ignore (like a dynamic timestamp) and create a Calabash Element
timestamp_element = query("* marked:'timestamp'").first
calabash_timestamp = Applitools::Calabash::CalabashElement.new(timestamp_element)

# Ignore this element during the check
target = Applitools::Calabash::Target.new.ignore(calabash_timestamp)
eyes.check('Home Screen', target)

Debug Screenshots

For troubleshooting, you can enable debug screenshots:

eyes.debug_screenshots = true

This will save additional screenshots during the check process to help diagnose issues.

Device Pixel Ratio

The SDK automatically handles device pixel ratio (display density) on different devices, but you can also set it manually if needed:

eyes.device_pixel_ratio = 2.0

Integration with Cucumber

Step Definitions

The SDK provides pre-defined step definitions for use in your Cucumber features:

# Initialize Eyes
Given I use Eyes with app name "My App" and test name "My Test"

# Set the viewport size
And I set the viewport size to 320x480

# Check operations
Then I check the window with tag "Home Screen"
And I check the element "username_field" with tag "Username Field"
And I check the window fully with tag "Full Page"

# Close Eyes
And I close Eyes session

Hooks

You can use Cucumber hooks to set up and tear down Eyes sessions:

# env.rb or hooks.rb
Before do |scenario|
  @eyes = Applitools::Calabash::Eyes.new
  @eyes.api_key = ENV['APPLITOOLS_API_KEY']
end

After do |scenario|
  @eyes.abort_if_not_closed
end

RSpec Matchers

The SDK includes RSpec matchers for visual validation:

# Match a specific element against the baseline
expect(element).to visually_match('Username Field')

# Match the entire screen
expect(screen).to visually_match('Home Screen')

Platform-Specific Features

Android Specifics

The SDK includes specialized support for Android:

Automatic handling of Android's activity hierarchy
Support for ScrollView and ListView scrollable elements
Android-specific step definitions in Cucumber

iOS Specifics

For iOS, the SDK provides:

Support for UITableView and UIScrollView scrollable elements
Handling of iOS status bar and navigation bar
iOS-specific step definitions in Cucumber

Best Practices

API Key Management: Always use environment variables for your API key
```
eyes.api_key = ENV['APPLITOOLS_API_KEY']
```
Error Handling: Always include proper cleanup in Cucumber hooks
```
After do |scenario|
 @eyes.abort_if_not_closed
end
```

Meaningful Test Names: Use descriptive names for your tests and checks

eyes.open(app_name: 'Banking App', test_name: 'Login Flow - Android')
eyes.check('Login Form', target)

Batch Tests: Group related tests in a batch

batch = Applitools::BatchInfo.new('Login Flow Tests')
eyes.batch = batch

Scrollable Content: When testing scrollable content, use the fully method
```
eyes.check('Feed Screen', Applitools::Calabash::Target.new.fully)
```
Stable Identifiers: Use stable identifiers for elements in your application to ensure consistent testing
```
# Good - uses a stable identifier
element = query("* marked:'login_button'").first
```

# Avoid - potentially unstable element = query("button index:0").first


7. **Viewport Configuration**: Set a consistent viewport size for tests
   ```ruby
   eyes.open(
     app_name: 'My App',
     test_name: 'My Test',
     viewport_size: { width: 375, height: 667 }
   )

Common Pitfalls

Missing API Key: Forgetting to set the API key

# This must be set before calling eyes.open
eyes.api_key = ENV['APPLITOOLS_API_KEY']

Resource Cleanup: Failing to close Eyes or abort if an exception occurs

# Always include in After hook or ensure block
eyes.abort_if_not_closed

Element Not Found: Trying to check elements that don't exist

# Always check that your queries return results
elements = query("* marked:'login_button'")
if elements.any?
 calabash_element = Applitools::Calabash::CalabashElement.new(elements.first)
 eyes.check('Login Button', Applitools::Calabash::Target.new.region(calabash_element))
else
 puts "Warning: Login button not found"
end

Scrollable Content Issues: Not waiting for scrolling to complete

# Always ensure animations have completed before visual checks
wait_for_none_animating
eyes.check('Scrollable Content', Applitools::Calabash::Target.new.fully)

Device Orientation: Inconsistent device orientation between tests

# Set a consistent orientation before visual testing
rotate_device("portrait")
eyes.check('Home Screen', Applitools::Calabash::Target.new)

Custom Screenshot Handling: Working with screenshots

# Recommended: Use public API methods for screenshot handling
eyes.check('Home Screen', Applitools::Calabash::Target.new)

# Avoid: Directly accessing internal implementation details # Use the public API for more stable behavior


## FAQ

### How do I set my API key?

```ruby
eyes.api_key = ENV['APPLITOOLS_API_KEY']

It's recommended to use environment variables rather than hardcoding API keys.

Can I use Eyes Calabash with both Android and iOS?

Yes, the SDK automatically detects which platform you're running on and adjusts its behavior accordingly. The same code works for both platforms.

How do I handle dynamic content that changes between test runs?

You can use the ignore regions feature:

# Find the dynamic element
dynamic_element = query("* marked:'timestamp'").first
calabash_dynamic = Applitools::Calabash::CalabashElement.new(dynamic_element)

# Ignore this element during checks
target = Applitools::Calabash::Target.new.ignore(calabash_dynamic)
eyes.check('Home Screen', target)

How do I check scrollable content like a feed or a long form?

Use the fully method:

eyes.check('Feed Screen', Applitools::Calabash::Target.new.fully)

This will automatically scroll through the content and stitch the screenshots together.

What if my app has custom scrollable containers?

The SDK supports common containers like Android ScrollView and iOS UITableView. For custom containers, you may need to implement your own scrolling in Calabash before visual checks.

How do I debug visual test failures?

Enable debug screenshots:
```
eyes.debug_screenshots = true
```
Check the Applitools dashboard for visual differences.

Use this code to log more information:

Applitools::EyesLogger.log_handler = Logger.new(STDOUT)
Applitools::EyesLogger.log_handler.level = Logger::DEBUG

How do I handle different device screen sizes and resolutions?

The SDK automatically handles device pixel ratios. For handling different screen sizes, you can use baseline branches:

# Set a branch name based on the device type
eyes.branch_name = "iPhone8"
# or
eyes.branch_name = "SamsungGalaxyS9"

Can I use Eyes Calabash with Calabash-Android and Calabash-iOS?

Yes, the SDK is designed to work with both platforms and will automatically detect which one you're using.

License

This SDK is distributed under the Applitools SDK License Agreement. See the LICENSE file for more details.

Important: This SDK may be used solely for your personal, non-commercial purposes. For commercial use, please contact your Applitools representative or visit applitools.com to obtain a commercial license.