Charai

Chat + Ruby + AI = Charai

Charai is an AI-powered Capybara driver that enables natural language web testing. Write your E2E tests by describing what you want to test in plain language, and let AI handle the browser interactions.

Requirements

Ruby 2.7 or higher
Firefox Developer Edition
OpenAI API key or compatible AI service (Azure OpenAI, Gemini, Ollama)

Setup

Add gem 'charai' into your project's Gemfile, and then bundle install

Also, this gem requires Firefox Developer Edition to be installed on the location below:

/Applications/Firefox Developer Edition.app (macOS)
/usr/bin/firefox-devedition (Linux)

Configuration

Configure your Capybara driver like below.

Capybara.register_driver :charai do |app|
  Charai::Driver.new(app, openai_configuration: config)
end

Capybara.register_driver :charai_headless do |app|
  Charai::Driver.new(app, openai_configuration: config, headless: true)
end

Please note that this driver required OpenAI service.

OpenAI

config = Charai::OpenaiConfiguration.new(
  model: 'gpt-4o',
  api_key: 'sk-xxxxxxxxxxx'
)

Azure OpenAI (Recommended)

config = Charai::AzureOpenaiConfiguration.new(
  endpoint_url: 'https://YOUR-APP.openai.azure.com/openai/deployments/gpt-4o/chat/completions?api-version=2024-05-01-preview',
  api_key: 'aabbcc00112233445566'
)

⚠️ Important Note for Azure OpenAI Environment Variables:

When setting the endpoint URL via environment variables, you must include the full path with /deployments/{model}/chat/completions:

# ❌ This will NOT work
export AZURE_OPENAI_ENDPOINT_URL="https://your-resource.cognitiveservices.azure.com/openai/responses?api-version=2025-04-01-preview"

# ✅ This works correctly
export AZURE_OPENAI_ENDPOINT_URL="https://your-resource.cognitiveservices.azure.com/openai/deployments/gpt-4o-mini/chat/completions?api-version=2025-04-01-preview"

The endpoint URL must specify:

Your Azure resource endpoint
The deployment name (e.g., gpt-4o-mini, gpt-4o)
The /chat/completions path
The API version parameter

Gemini

config = Charai::GeminiOpenaiConfiguration.new(
  model: 'gemini-2.0-flash-exp',
  api_key: 'your-gemini-api-key'
)

Ollama (Local AI)

config = Charai::OllamaConfiguration.new(
  endpoint_url: 'http://localhost:11434',
  model: 'llama3.2'
)

Usage

Basic Example

RSpec.describe "Login test", type: :feature do
  before do
    Capybara.current_driver = :charai
  end

  it "should login successfully" do
    page.driver << "    * Navigate to the login page\n    * Enter \"[email protected]\" in the email field\n    * Enter \"password123\" in the password field\n    * Click the login button\n    * Verify that you're redirected to the dashboard page\n    * Check that the username \"John Doe\" is displayed in the header\n    MARKDOWN\n  end\nend\n"

Advanced Example with Instructions

You can provide additional context about your application to help the AI better understand the page structure:

before do
  Capybara.current_driver    = :charai
  Capybara.javascript_driver = :charai

  page.driver.additional_instruction = "  * \u3053\u306E\u30DA\u30FC\u30B8\u306F\u30013\u30DA\u30A4\u30F3\u69CB\u9020\u3067\u3059\u3002\u30E6\u30FC\u30B6\u304C\u4ED5\u4E8B\u3092\u63A2\u3059\u305F\u3081\u306E\u30DA\u30FC\u30B8\u3067\u3059\u3002\n  * \u5DE6\u30DA\u30A4\u30F3\u306B\u306F\u4ED5\u4E8B\u306E\u7D5E\u308A\u8FBC\u307F\u304C\u3067\u304D\u308B\u30D5\u30A3\u30EB\u30BF\u30FC\u3001\u4E2D\u592E\u30DA\u30A4\u30F3\u304C\u4ED5\u4E8B\uFF08\u6C42\u4EBA\uFF09\u306E\u4E00\u89A7\u3067\u300130\u4EF6\u305A\u3064\u8868\u793A\u3055\u308C\u307E\u3059\u3002\n  * \u5DE6\u30DA\u30A4\u30F3\u306B\u30DE\u30A6\u30B9\u3092\u7F6E\u3044\u3066\u30B9\u30AF\u30ED\u30FC\u30EB\u3057\u3066\u3082\u3001\u4E2D\u592E\u30DA\u30A4\u30F3\u306F\u30B9\u30AF\u30ED\u30FC\u30EB\u3055\u308C\u307E\u305B\u3093\u3002\u4E00\u89A7\u3092\u30B9\u30AF\u30ED\u30FC\u30EB\u3057\u305F\u3044\u3068\u304D\u306B\u306F\u3001\u4E2D\u592E\u30DA\u30A4\u30F3\u306E\u5EA7\u6A19\u3092\u78BA\u8A8D\u3057\u3001\u305D\u306E\u4E2D\u592E\u306B\u30DE\u30A6\u30B9\u3092\u7F6E\u3044\u3066\u30B9\u30AF\u30ED\u30FC\u30EB\u3057\u3066\u304F\u3060\u3055\u3044\u3002\n  * \u53F3\u30DA\u30A4\u30F3\u306F\u3001\u5E83\u544A\u30A8\u30EA\u30A2\u3067\u3059\u3002\u691C\u7D22\u6761\u4EF6\u306B\u5FDC\u3058\u305F\u5E83\u544A\u304C\u8868\u793A\u3055\u308C\u307E\u3059\u3002\n  MARKDOWN\nend\n\nit 'should work' do\n  page.driver << <<~MARKDOWN\n  * \u4ED5\u4E8B\u306E\u4E00\u89A7\u304C\u8868\u793A\u3055\u308C\u305F\u3089\u3001\u5DE6\u30DA\u30A4\u30F3\u3067\u30B5\u30FC\u30D0\u30FC\u30B5\u30A4\u30C9\u30A8\u30F3\u30B8\u30CB\u30A2\u3067\u300CRuby on Rails\u300D\u306E\u4ED5\u4E8B\u306B\u7D5E\u308A\u8FBC\u307F\u3092\u3057\u3066\u304F\u3060\u3055\u3044\u3002\n  * \u5DE6\u30DA\u30A4\u30F3\u3067\u7D5E\u308A\u8FBC\u3093\u3060\u3089\u3001\u4E2D\u592E\u30DA\u30A4\u30F3\u306BRuby on Rails\u306B\u95A2\u3059\u308B\u4ED5\u4E8B\u304C\u8868\u793A\u3055\u308C\u3066\u3044\u308B\u3053\u3068\u3092\u78BA\u8A8D\u3057\u3066\u304F\u3060\u3055\u3044\u3002\n  * Ruby on Rails\u306B\u95A2\u4FC2\u306E\u306A\u3044\u4ED5\u4E8B\u304C\u3001\u691C\u7D22\u7D50\u679C\u4EF6\u6570\u306E\u534A\u5206\u4EE5\u4E0A\u3042\u308B\u3070\u3042\u3044\u306B\u306F\u3001\u3053\u306E\u30C6\u30B9\u30C8\u300C\u691C\u7D22\u7D50\u679C\u4E0D\u9069\u5408\u300D\u3068\u3057\u3066\u5931\u6557\u3068\u3057\u3066\u304F\u3060\u3055\u3044\u3002\n  MARKDOWN\nend\n"

Report for long-running E2E tests

We often trigger E2E test during the night since it costs a lot of time. It is really boring to sit down in front of the PC during E2E testing.

This driver provides an extension (callback) feature for reporting.

First, let's prepare a report formatter like this.

class HtmlReport
  def initialize
    @content = []
  end

  def start(introduction)
    @content << "    <html>\n    <head><link rel=\"stylesheet\" href=\"https://cdnjs.cloudflare.com/ajax/libs/mini.css/3.0.1/mini-default.min.css\"></head>\n    <body>\n    <pre style=\"margin: 100px 75px; border-left: 0px;\">\#{introduction}</pre>\n    HTML\n  end\n\n  def add_conversation(content, answer)\n    if content.is_a?(Array)\n      text = content.find { |c| c[:type] == 'text' }[:text]\n\n      @content << <<~HTML\n      <div style=\"margin: 40px 25px\" class=\"card fluid\">\n      <h4 style=\"border-left: .25rem solid var(--pre-color);\">user</h4>\n      <pre>\#{text}</pre>\n      HTML\n\n      content.each do |c|\n        next unless c[:type] == 'image_url'\n\n        @content << <<~HTML\n        <img src=\"\#{c[:image_url][:url]}\" width=\"480\" />\n        HTML\n      end\n\n      @content << <<~HTML\n      <h4 style=\"text-align: end; border-right: .25rem solid var(--pre-color);\">assistant</h4>\n      <pre style=\"border-left: 0px; border-right: .25rem solid var(--pre-color);\">\#{answer}</pre>\n      </div>\n      HTML\n    else\n      @content << <<~HTML\n      <div style=\"margin: 40px 25px\" class=\"card fluid\">\n      <h4 style=\"border-left: .25rem solid var(--pre-color);\">user</h4>\n      <pre>\#{content}</pre>\n\n      <h4 style=\"text-align: end; border-right: .25rem solid var(--pre-color);\">assistant</h4>\n      <pre style=\"border-left: 0px; border-right: .25rem solid var(--pre-color);\">\#{answer}</pre>\n      </div>\n      HTML\n    end\n  end\n\n  def to_html\n    @content << \"</body></html>\"\n    @content.join(\"\\n\")\n  end\nend\n"

and then configure a callback for recording test-results into the report. It would be a good choice to use Allure reports's attachment feature for this.

config.around(:each, type: :feature) do |example|
  report = HtmlReport.new
  Capybara.current_session.driver.callback = {
    on_chat_start: -> (introduction) {
      report.start(introduction)
    },
    on_chat_conversation: ->(content_hash, answer) {
      puts answer
      report.add_conversation(content_hash, answer)
    },
  }

  example.run

  Allure.add_attachment(
    name: "Chat Report",
    source: report.to_html,
    type: 'text/html',
  )
end

With this report, we can check evidences for each test and investigate failed tests (postmortem).

Troubleshooting

Firefox Developer Edition not found

Make sure Firefox Developer Edition is installed in the correct location:

macOS: /Applications/Firefox Developer Edition.app
Linux: /usr/bin/firefox-devedition

You can also set a custom path:

Charai::Driver.new(app,
  openai_configuration: config,
  firefox_path: '/path/to/firefox-dev'
)

AI not understanding the page

Provide more context using additional_instruction:

page.driver.additional_instruction = "* This is a single-page application (SPA)\n* Wait for elements to load before interacting\n* The main navigation is in a hamburger menu on mobile\n"

Tests running slowly

Consider using a faster AI model or running in headless mode:

Capybara.current_driver = :charai_headless

License

The gem is available as open source under the terms of the MIT License.