Class: Aidp::Harness::AIFilterFactory

Inherits:
Object
  • Object
show all
Defined in:
lib/aidp/harness/ai_filter_factory.rb

Overview

AI-powered factory for generating deterministic filter definitions

Uses AI ONCE during configuration to analyze a tool and generate regex patterns and extraction rules. The generated FilterDefinition is then applied deterministically at runtime without any AI calls.

Examples:

Generate a filter for pytest

factory = AIFilterFactory.new(config)
definition = factory.generate_filter(
  tool_name: "pytest",
  tool_command: "pytest -v",
  sample_output: "... actual pytest output ..."
)
# Save definition to config, use deterministically at runtime

See Also:

Constant Summary collapse

GENERATION_PROMPT = 
"Analyze the following tool output and generate regex patterns for filtering.\nThe goal is to extract ONLY the important information (failures, errors, locations)\nand filter out noise, so that AI assistants receive concise, actionable output.\n\nTool: {{tool_name}}\nCommand: {{tool_command}}\n\nSample output:\n```\n{{sample_output}}\n```\n\nGenerate a filter definition with these components:\n\n1. summary_patterns: Regex patterns that match summary/result lines (e.g., \"5 passed, 2 failed\")\n2. failure_section_start: Regex pattern marking where failures section begins (if applicable)\n3. failure_section_end: Regex pattern marking where failures section ends (if applicable)\n4. error_section_start: Regex pattern for errors section start (if different from failures)\n5. error_section_end: Regex pattern for errors section end\n6. error_patterns: Regex patterns that identify error/failure indicator lines\n7. location_patterns: Regex patterns to extract file:line locations from output\n8. noise_patterns: Regex patterns for lines that should be filtered out (timestamps, progress bars, etc.)\n9. important_patterns: Regex patterns for lines that should ALWAYS be kept\n\nImportant guidelines for patterns:\n- Use simple, portable regex syntax\n- Escape special characters properly (dots, brackets, etc.)\n- Make patterns case-insensitive where appropriate\n- For location patterns, use capture groups to extract the file:line portion\n- Leave fields as null/empty if not applicable to this tool\n\nRespond with ONLY valid JSON matching this structure:\n{\n  \"tool_name\": \"string\",\n  \"summary_patterns\": [\"pattern1\", \"pattern2\"],\n  \"failure_section_start\": \"pattern or null\",\n  \"failure_section_end\": \"pattern or null\",\n  \"error_section_start\": \"pattern or null\",\n  \"error_section_end\": \"pattern or null\",\n  \"error_patterns\": [\"pattern1\"],\n  \"location_patterns\": [\"pattern with (capture) group\"],\n  \"noise_patterns\": [\"pattern1\"],\n  \"important_patterns\": [\"pattern1\"]\n}\n"
RESPONSE_SCHEMA =

JSON schema for validating AI response

{
  type: "object",
  properties: {
    tool_name: {type: "string"},
    summary_patterns: {type: "array", items: {type: "string"}},
    failure_section_start: {type: ["string", "null"]},
    failure_section_end: {type: ["string", "null"]},
    error_section_start: {type: ["string", "null"]},
    error_section_end: {type: ["string", "null"]},
    error_patterns: {type: "array", items: {type: "string"}},
    location_patterns: {type: "array", items: {type: "string"}},
    noise_patterns: {type: "array", items: {type: "string"}},
    important_patterns: {type: "array", items: {type: "string"}}
  },
  required: ["tool_name", "summary_patterns"]
}.freeze

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(config, provider_factory: nil) ⇒ AIFilterFactory

Initialize the AI filter factory

Parameters:

  • config (Configuration) —

    AIDP configuration

  • provider_factory (ProviderFactory, nil) (defaults to: nil) —

    Optional factory for AI providers



99
100
101
102
 
# File 'lib/aidp/harness/ai_filter_factory.rb', line 99

def initialize(config, provider_factory: nil)
  @config = config
  @provider_factory = provider_factory || ProviderFactory.new(config)
end

Instance Attribute Details

#config ⇒ Object (readonly)

Returns the value of attribute config.



93
94
95
 
# File 'lib/aidp/harness/ai_filter_factory.rb', line 93

def config
  @config
end

#provider_factory ⇒ Object (readonly)

Returns the value of attribute provider_factory.



93
94
95
 
# File 'lib/aidp/harness/ai_filter_factory.rb', line 93

def provider_factory
  @provider_factory
end

Instance Method Details

#generate_filter(tool_name:, tool_command:, sample_output: nil, tier: "mini") ⇒ FilterDefinition

Generate a filter definition for a tool

Parameters:

  • tool_name (String) —

    Human-readable tool name

  • tool_command (String) —

    The command used to run the tool

  • sample_output (String, nil) (defaults to: nil) —

    Sample output from the tool (for better patterns)

  • tier (String) (defaults to: "mini") —

    AI tier to use (“mini”, “standard”, “advanced”)

Returns:

Raises:



112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
 
# File 'lib/aidp/harness/ai_filter_factory.rb', line 112

def generate_filter(tool_name:, tool_command:, sample_output: nil, tier: "mini")
  Aidp.log_info("ai_filter_factory", "Generating filter definition",
    tool_name: tool_name, tool_command: tool_command, tier: tier)

  # Build prompt with context
  prompt = build_prompt(tool_name, tool_command, sample_output)

  # Get AI model for the tier
  thinking_manager = ThinkingDepthManager.new(config)
  provider_name, model_name, _model_data = thinking_manager.select_model_for_tier(
    tier,
    provider: config.respond_to?(:default_provider) ? config.default_provider : nil
  )

  Aidp.log_debug("ai_filter_factory", "Using AI model",
    provider: provider_name, model: model_name)

  # Call AI
  response = call_ai(provider_name, model_name, prompt)

  # Parse and validate response
  definition_data = parse_response(response)
  validate_patterns(definition_data)

  # Create FilterDefinition
  definition = FilterDefinition.new(
    tool_name: definition_data[:tool_name] || tool_name,
    tool_command: tool_command,
    summary_patterns: definition_data[:summary_patterns] || [],
    failure_section_start: definition_data[:failure_section_start],
    failure_section_end: definition_data[:failure_section_end],
    error_section_start: definition_data[:error_section_start],
    error_section_end: definition_data[:error_section_end],
    error_patterns: definition_data[:error_patterns] || [],
    location_patterns: definition_data[:location_patterns] || [],
    noise_patterns: definition_data[:noise_patterns] || [],
    important_patterns: definition_data[:important_patterns] || [],
    context_lines: 3
  )

  Aidp.log_info("ai_filter_factory", "Filter definition generated",
    tool_name: definition.tool_name,
    summary_pattern_count: definition.summary_patterns.size,
    location_pattern_count: definition.location_patterns.size)

  definition
rescue => e
  Aidp.log_error("ai_filter_factory", "Failed to generate filter",
    tool_name: tool_name, error: e.message, error_class: e.class.name)
  raise GenerationError, "Failed to generate filter for #{tool_name}: #{e.message}"
end

#generate_from_command(tool_command:, project_dir: Dir.pwd, tier: "mini") ⇒ FilterDefinition

Generate filter from tool command by running it and capturing output

Parameters:

  • tool_command (String) —

    Command to run

  • project_dir (String) (defaults to: Dir.pwd) —

    Directory to run command in

  • tier (String) (defaults to: "mini") —

    AI tier to use

Returns:



170
171
172
173
174
175
176
177
178
179
180
181
182
 
# File 'lib/aidp/harness/ai_filter_factory.rb', line 170

def generate_from_command(tool_command:, project_dir: Dir.pwd, tier: "mini")
  tool_name = extract_tool_name(tool_command)

  # Try to get sample output by running the command
  sample_output = capture_sample_output(tool_command, project_dir)

  generate_filter(
    tool_name: tool_name,
    tool_command: tool_command,
    sample_output: sample_output,
    tier: tier
  )
end