Module: Appium::Core::Base::Device::ImageComparison

Included in:
Bridge::MJSONWP, Bridge::W3C
Defined in:
lib/appium_lib_core/common/device/image_comparison.rb

Constant Summary collapse

MODE = 
[:matchFeatures, :getSimilarity, :matchTemplate].freeze
MATCH_FEATURES = 
{
  detector_name: %w(AKAZE AGAST BRISK FAST GFTT KAZE MSER SIFT ORB),
  match_func: %w(FlannBased BruteForce BruteForceL1 BruteForceHamming BruteForceHammingLut BruteForceSL2),
  goodMatchesFactor: nil, # Integer
  visualize: [true, false]
}.freeze
MATCH_TEMPLATE = 
{
  visualize: [true, false]
}.freeze
GET_SIMILARITY = 
{
  visualize: [true, false]
}.freeze

Instance Method Summary collapse

Instance Method Details

#compare_images(mode: :matchFeatures, first_image:, second_image:, options: nil) ⇒ Hash

Performs images comparison using OpenCV framework features. It is expected that both OpenCV framework and opencv4nodejs module are installed on the machine where Appium server is running.

Parameters:

  • mode (Symbol) (defaults to: :matchFeatures) —

    One of possible comparison modes: ‘:matchFeatures`, `:getSimilarity`, `:matchTemplate`. `:matchFeatures is by default.

  • first_image (String) —

    An image data. All image formats, that OpenCV library itself accepts, are supported.

  • second_image (String) —

    An image data. All image formats, that OpenCV library itself accepts, are supported.

  • options (Hash) (defaults to: nil) —

    The content of this dictionary depends on the actual ‘mode` value. See the documentation on `appium-support` module for more details.

Returns:

  • (Hash) —

    The content of the resulting dictionary depends on the actual ‘mode` and `options` values. See the documentation on `appium-support` module for more details.



152
153
154
155
156
157
158
159
160
161
162
 
# File 'lib/appium_lib_core/common/device/image_comparison.rb', line 152

def compare_images(mode: :matchFeatures, first_image:, second_image:, options: nil)
  raise "content_type should be #{MODE}" unless MODE.member?(mode)

  params = {}
  params[:mode] = mode
  params[:firstImage] = Base64.strict_encode64 first_image
  params[:secondImage] = Base64.strict_encode64 second_image
  params[:options] = options if options

  execute(:compare_images, {}, params)
end

#find_image_occurrence(full_image:, partial_image:, visualize: false, threshold: nil) ⇒ Object

Performs images matching by template to find possible occurrence of the partial image in the full image with default options. Read template_matching for more details on this topic.

Examples:

@driver.find_image_occurrence full_image: "image data 1", partial_image: "image data 2"

visual = @@driver.find_image_occurrence full_image: image1, partial_image: image2, visualize: true
File.write 'find_result_visual.png', Base64.decode64(visual['visualization']) # if the image is PNG

Parameters:

  • full_image (String) —

    A full image data.

  • partial_image (String) —

    A partial image data. All image formats, that OpenCV library itself accepts, are supported.

  • visualize (Bool) (defaults to: false) —

    Makes the endpoint to return an image, which contains the visualized result of the corresponding picture matching operation. This option is disabled by default.

  • threshold (Float) (defaults to: nil) —
    0.5

    At what normalized threshold to reject



105
106
107
108
109
110
111
112
113
 
# File 'lib/appium_lib_core/common/device/image_comparison.rb', line 105

def find_image_occurrence(full_image:, partial_image:, visualize: false, threshold: nil)
  raise "visualize should be #{MATCH_TEMPLATE[:visualize]}" unless MATCH_TEMPLATE[:visualize].member?(visualize)

  options = {}
  options[:visualize] = visualize
  options[:threshold] = threshold unless threshold.nil?

  compare_images(mode: :matchTemplate, first_image: full_image, second_image: partial_image, options: options)
end

#get_images_similarity(first_image:, second_image:, visualize: false) ⇒ Object

Performs images matching to calculate the similarity score between them with default options. The flow there is similar to the one used in ‘find_image_occurrence` but it is mandatory that both images are of equal size.

Examples:

@driver.get_images_similarity first_image: "image data 1", second_image: "image data 2"

visual = @@driver.get_images_similarity first_image: image1, second_image: image2, visualize: true
File.write 'images_similarity_visual.png', Base64.decode64(visual['visualization']) # if the image is PNG

Parameters:

  • first_image (String) —

    An image data. All image formats, that OpenCV library itself accepts, are supported.

  • second_image (String) —

    An image data. All image formats, that OpenCV library itself accepts, are supported.

  • visualize (Bool) (defaults to: false) —

    Makes the endpoint to return an image, which contains the visualized result of the corresponding picture matching operation. This option is disabled by default.



130
131
132
133
134
135
136
137
 
# File 'lib/appium_lib_core/common/device/image_comparison.rb', line 130

def get_images_similarity(first_image:, second_image:, visualize: false)
  raise "visualize should be #{GET_SIMILARITY[:visualize]}" unless GET_SIMILARITY[:visualize].member?(visualize)

  options = {}
  options[:visualize] = visualize

  compare_images(mode: :getSimilarity, first_image: first_image, second_image: second_image, options: options)
end

#match_images_features(first_image:, second_image:, detector_name: 'ORB', match_func: 'BruteForce', good_matches_factor: nil, visualize: false) ⇒ Object

Performs images matching by features with default options. Read py_matcher for more details on this topic.

Examples:

@driver.match_images_features first_image: "image data 1", second_image: "image data 2"

visual = @@driver.match_images_features first_image: image1, second_image: image2, visualize: true
File.write 'match_images_visual.png', Base64.decode64(visual['visualization']) # if the image is PNG

Parameters:

  • first_image (String) —

    An image data. All image formats, that OpenCV library itself accepts, are supported.

  • second_image (String) —

    An image data. All image formats, that OpenCV library itself accepts, are supported.

  • detector_name (String) (defaults to: 'ORB') —

    Sets the detector name for features matching algorithm. Some of these detectors (FAST, AGAST, GFTT, FAST, SIFT and MSER) are not available in the default OpenCV installation and have to be enabled manually before library compilation. The default detector name is ‘ORB’.

  • match_func (String) (defaults to: 'BruteForce') —

    The name of the matching function. The default one is ‘BruteForce’.

  • good_matches_factor (String) (defaults to: nil) —

    The maximum count of “good” matches (e. g. with minimal distances). The default one is nil.

  • visualize (Bool) (defaults to: false) —

    Makes the endpoint to return an image, which contains the visualized result of the corresponding picture matching operation. This option is disabled by default.



61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
 
# File 'lib/appium_lib_core/common/device/image_comparison.rb', line 61

def match_images_features(first_image:,
                          second_image:,
                          detector_name: 'ORB',
                          match_func: 'BruteForce',
                          good_matches_factor: nil,
                          visualize: false)
  unless MATCH_FEATURES[:detector_name].member?(detector_name.to_s)
    raise "detector_name should be #{MATCH_FEATURES[:detector_name]}"
  end

  unless MATCH_FEATURES[:match_func].member?(match_func.to_s)
    raise "match_func should be #{MATCH_FEATURES[:match_func]}"
  end

  raise "visualize should be #{MATCH_FEATURES[:visualize]}" unless MATCH_FEATURES[:visualize].member?(visualize)

  options = {}
  options[:detectorName] = detector_name.to_s.upcase
  options[:matchFunc] = match_func.to_s
  options[:goodMatchesFactor] = good_matches_factor.to_i unless good_matches_factor.nil?
  options[:visualize] = visualize

  compare_images(mode: :matchFeatures, first_image: first_image, second_image: second_image, options: options)
end