Module: PWN::AI::Agent::Learning

Defined in:

lib/pwn/ai/agent/learning.rb

Overview

PWN::AI::Agent::Learning is the self-improvement engine that closes the pwn-ai feedback loop. It captures task outcomes, mines session transcripts for durable lessons, promotes successful workflows into reusable skills, and keeps ~/.pwn lean (memory + learning.jsonl + mistakes + sessions) so the agent gets sharper over time instead of accumulating noise.

Data flows:

Loop.run --(tool telemetry)--> Metrics.record
Loop.run --(final answer)----> Learning.auto_introspect (opt-in)
auto_introspect --(throttled)--> Learning.gc_stores!  # ~/.pwn lean
model    --(tool calls)------> learning_note_outcome / _distill_skill
PromptBuilder <----------------- Learning.to_context + Metrics.to_context

Everything is file-backed under ~/.pwn so it survives across REPL restarts and is shared by every future session.

Constant Summary

LEARNING_FILE =

File.join(Dir.home, '.pwn', 'learning.jsonl')

DISPUTED_FILE =

File.join(Dir.home, '.pwn', 'learning', 'disputed.jsonl')

LESSONS_FILE =

File.join(Dir.home, '.pwn', 'lessons.json')

FINETUNE_DIR =

File.join(Dir.home, '.pwn', 'finetune')

INTROSPECT_SOFT_MS =

P0 — post-answer introspect must not train "stop early" while spending the iteration budget after the final. Soft cap skips expensive stages (tool critic, PRM-LLM, reflect, extrospect); hard cap keeps only note_outcome + judge(heuristic) + sentinel.

2_500

INTROSPECT_HARD_MS =

8_000

INTROSPECT_MIN_STAGES =

i[judge note_outcome fold_judge sentinel].freeze

MAX_MEMORY_ENTRIES =

200

MAX_OUTCOME_ROWS =

Lean outcome retention — keep gold RL signal, drop bulk auto noise.

800

OUTCOME_RETAIN_DAYS =

45

OUTCOME_RECENT_DAYS =

14

OUTCOME_DETAILS_MAX =

800

GOLD_MIN_SCORE =

0.6

EXEMPLARS_POOL_MIN =

200

FAILURE_WINDOW_MIN =

200

HIGH_VALUE_TAGS =

%w[
  needs_human extro_verify sdr gqrx rl pwn-ai curriculum hindsight her
].freeze

LOW_VALUE_ONLY_TAGS =

%w[
  auto loop partial wrong solved offline_judge plan_cover_high
].freeze

PRUNE_EVERY_N_APPENDS =

25

CLAIM_METRIC_WORDS =

E3/P26 — only CVE-ids or software-name + full semver (x.y.z). Two-part floats ("cap 0.2", "proxy 1.0", "judge 37.0") are RL metric crumbs that were scraped by verify_as_reward and flooded learning.jsonl with extro_verify :unknown failures.

%w[
  cap share proxy judge success only now clears gap score rate mean
  brier overconf distrust trajectory_fraction handler orm prm delta
  limit window pct percent ms iter budget conf confidence n
  ruby python linux kernel host cwd e.g e.g.
].freeze

CLAIM_RX =

/
  CVE-\d{4}-\d{4,7}
  |
  \b
  (?!
    (?:cap|share|proxy|judge|success|only|now|clears|gap|score|rate|mean|
       brier|overconf|distrust|trajectory_fraction|handler|orm|prm|delta|
       limit|window|pct|percent|ms|iter|budget|conf|confidence|
       ruby|python|linux|kernel|host|cwd|e\.g)
    \b
  )
  [A-Za-z][\w.+-]{2,}
  \s+
  v?\d+\.\d+\.\d+(?:[-+][\w.]+)?
  \b
/x

SFT_MIN_SCORE =

P12 — SFT quality gate (as hard as DPO source-cap): drop HER/soft, low judge_score, auto-only noise without score, and PRM-compress trajectories so LoRA is not 5MB of "how we flailed".

0.6

SFT_MAX_TOOL_CHARS =

1_200

PROCESS_SOP_RX =

M4.1 — keyword gate for "process / hygiene" SOPs the operator keeps re-requesting (rubocop, rake, rspec, conventions). These must become durable Memory lessons, not just learning.jsonl rows.

%r{\b(rubocop|rake\b|rspec|bundle\s+exec|conventions?|lint(?:ing)?|style/|code\s*hygiene|post[- ]?patch|after\s+(?:every\s+)?(?:patch|change|edit))\b}i

FAILURE_FINAL_RX =

/\[pwn-ai\] (iteration budget exhausted|engine returned no message)|\b(i (was )?unable to|i could not|i couldn'?t|cannot proceed|failed to)\b/i

Class Method Summary

• .authors ⇒ Object

  Author(s)

  0day Inc.

• .auto_introspect(opts = {}) ⇒ Object

  Supported Method Parameters

  PWN::AI::Agent::Learning.auto_introspect( session_id: 'required - id of the just-completed session', request: 'optional - original user request (for outcome logging)', final: 'optional - final assistant answer (for outcome logging)' ).

• .compact!(opts = {}) ⇒ Object
• .consistency_check(opts = {}) ⇒ Object
• .consolidate(opts = {}) ⇒ Object

  Supported Method Parameters

  removed = PWN::AI::Agent::Learning.consolidate( max_entries: 'optional - hard cap on PWN::Memory size (default MAX_MEMORY_ENTRIES)' ).

• .disputed_save(opts = {}) ⇒ Object
• .distill_skill(opts = {}) ⇒ Object

  Supported Method Parameters

  skill = PWN::AI::Agent::Learning.distill_skill( name: 'required - snake_case name for the new skill', session_id: 'optional - PWN::Sessions id to mine (uses its transcript)', content: 'optional - explicit markdown body; overrides transcript mining', references: 'optional - Array of reference URLs / CWE / CVE / ATT&CK ids' ).

• .exemplars_for(opts = {}) ⇒ Object

  Supported Method Parameters

  msgs = PWN::AI::Agent::Learning.exemplars_for( request: 'required - current user request', limit: 'optional - max exemplar traces to return (default 1)', max_msgs: 'optional - cap on messages per exemplar (default 6)' ).

• .export_finetune(opts = {}) ⇒ Object
• .flip_last_outcome(opts = {}) ⇒ Object
• .gc_stores!(opts = {}) ⇒ Object

  One-shot lean across memory + learning + mistakes + sessions.

• .help ⇒ Object

  Display Usage for this Module.

• .lean!(opts = {}) ⇒ Object

  Memory lean + outcome prune.

• .lesson_observe(opts = {}) ⇒ Object
• .lesson_prompt(opts = {}) ⇒ Object
• .lesson_record(opts = {}) ⇒ Object
• .list_conflicted(opts = {}) ⇒ Object
• .note_outcome(opts = {}) ⇒ Object

  Supported Method Parameters

  entry = PWN::AI::Agent::Learning.note_outcome( task: 'required - short description of what was attempted', success: 'required - Boolean, did the attempt achieve its goal', details: 'optional - free-form notes / error / evidence', session_id: 'optional - PWN::Sessions id this outcome belongs to', tags: 'optional - Array of String labels for later retrieval' ).

• .outcomes(opts = {}) ⇒ Object

  Supported Method Parameters

  rows = PWN::AI::Agent::Learning.outcomes( limit: 'optional - max entries returned newest-first (default 50)', success: 'optional - filter by Boolean outcome', tag: 'optional - filter by tag substring' ).

• .prune_outcomes!(opts = {}) ⇒ Object

  Supported Method Parameters

  result = PWN::AI::Agent::Learning.prune_outcomes!( dry_run: 'optional - Boolean (default false)', max_rows: 'optional - hard cap (default MAX_OUTCOME_ROWS)', retain_days: 'optional - age floor for low-value drop', recent_days: 'optional - always keep newer than this' ).

• .purge_noise ⇒ Object

  Supported Method Parameters

  PWN::AI::Agent::Learning.purge_noise.

• .reconcile_verdict_tags!(opts = {}) ⇒ Object

  One-shot / on-load repair: rewrite tags+details where verdict label disagrees with score (solved @ 0.3 etc.).

• .reflect(opts = {}) ⇒ Object

  Supported Method Parameters

  report = PWN::AI::Agent::Learning.reflect( session_id: 'required - PWN::Sessions id to analyse', dry_run: 'optional - when true, do not write to Memory/Skills (default false)' ).

• .requeue_conflicted(opts = {}) ⇒ Object
• .reset ⇒ Object

  Supported Method Parameters

  PWN::AI::Agent::Learning.reset.

• .stats ⇒ Object

  Supported Method Parameters

  stats = PWN::AI::Agent::Learning.stats.

• .to_context(opts = {}) ⇒ Object

  Supported Method Parameters

  ctx = PWN::AI::Agent::Learning.to_context( limit: 'optional - number of recent outcomes to surface (default 5)' ).

• .update_skill(opts = {}) ⇒ Object

  Fold RL artefacts (mistakes / structured_fix / an explicit lesson) into an existing skill.

Class Method Details

.authors ⇒ Object

Author(s)

0day Inc. support@0dayinc.com

# File 'lib/pwn/ai/agent/learning.rb', line 1960

public_class_method def self.authors
  "AUTHOR(S):\n  0day Inc. <support@0dayinc.com>\n"
end

.auto_introspect(opts = {}) ⇒ Object

Supported Method Parameters

PWN::AI::Agent::Learning.auto_introspect( session_id: 'required - id of the just-completed session', request: 'optional - original user request (for outcome logging)', final: 'optional - final assistant answer (for outcome logging)' )

Called by Loop.run when PWN::Env[:agent][:auto_introspect] is truthy. Never raises — learning must not break the primary loop.

# File 'lib/pwn/ai/agent/learning.rb', line 594

public_class_method def self.auto_introspect(opts = {})
  session_id = opts[:session_id]
  return unless session_id
  return unless auto_introspect_enabled?

  # Hermes split: Loop.run is on the stack → defer critic/judge/PRM/HER
  # onto a daemon thread so the user already has the reply. Specs,
  # cron, and tools pass through (inline:true or no user-path depth).
  return TurnFinalizer.defer(opts.merge(inline: true)) if defined?(TurnFinalizer) && !opts[:inline] && TurnFinalizer.should_defer?

  t0 = Process.clock_gettime(Process::CLOCK_MONOTONIC)
  stages_run = []
  stages_skipped = []
  budget_hot = begin
    defined?(Loop) && Loop.respond_to?(:budget_exhaustion_hot?, true) &&
      Loop.send(:budget_exhaustion_hot?)
  rescue StandardError
    false
  end

  elapsed_ms = lambda do
    ((Process.clock_gettime(Process::CLOCK_MONOTONIC) - t0) * 1000).round
  end
  # soft = skip expensive; hard = stop almost everything
  over_soft = lambda do
    ms = elapsed_ms.call
    ms >= INTROSPECT_SOFT_MS || budget_hot
  end
  over_hard = lambda do
    ms = elapsed_ms.call
    ms >= INTROSPECT_HARD_MS
  end

  proxy_ok = infer_success(session_id: session_id, final: opts[:final])

  crit = { verdict: :pass, source: :skipped }
  # Live-turn critic is always text-only. A tool-armed persona is
  # another Loop.run of the same goal (opens more browsers, never
  # returns the operator answer). Cron/practice still call
  # Curriculum.critic without text_only.
  if !defined?(Curriculum) || over_hard.call
    stages_skipped << :critic
  else
    stages_run << :critic_text_only
    crit = Curriculum.critic(
      request: opts[:request],
      final: opts[:final],
      session_id: session_id,
      text_only: true
    )
  end

  # R1 judge — always attempt (heuristic is cheap; LLM gated inside)
  stages_run << :judge
  critic_pass = crit[:verdict] == :flaw ? false : nil
  v = Reward.judge(request: opts[:request], final: opts[:final], session_id: session_id, proxy_ok: proxy_ok, predicted: opts[:predicted], critic_pass: critic_pass) if defined?(Reward)
  v ||= { score: nil, source: :error, verdict: :unknown, success: nil }
  v = Reward.resolve_outcome(outcome: v, critic_pass: critic_pass)
  ok = v[:success]

  # W1 pending user_correction pair
  pend = Thread.current[:pwn_pending_pref]
  if pend && ok && defined?(Reward)
    stages_run << :user_correction_pref
    Reward.record_preference(
      prompt: pend[:prompt],
      rejected: pend[:rejected],
      chosen: opts[:final].to_s,
      source: :user_correction,
      shape: :revised_answer,
      force: true
    )
    Thread.current[:pwn_pending_pref] = nil
  end

  # Soft plan-quality feature (W3) — tag only; not full DPO.
  plan_cov = nil
  if defined?(Reward) && Reward.respond_to?(:plan_coverage)
    begin
      plan_for_cov = opts[:plan]
      unless plan_for_cov.nil?
        plan_cov = Reward.plan_coverage(
          plan: plan_for_cov || [],
          final: opts[:final],
          request: opts[:request],
          session_id: session_id
        )
        stages_run << :plan_coverage if plan_cov && plan_cov[:total].to_i.positive?
      end
    rescue StandardError => e
      warn "[pwn-ai/learning] plan_coverage swallowed: #{e.class}: #{e.message}"
    end
  end

  stages_run << :note_outcome
  outcome_tags = ['auto', 'loop', v[:verdict].to_s]
  outcome_tags << plan_cov[:tag] if plan_cov && plan_cov[:tag]
  outcome_tags << "plan_cover=#{plan_cov[:score]}" if plan_cov && plan_cov[:total].to_i.positive?
  # P29 — persist the bare user ask (strip REQUEST:/GOAL: envelopes at write time)
  task_txt = display_task(task: opts[:request].to_s)
  task_txt = opts[:request].to_s[0, 100] if task_txt.empty?
  note_outcome(
    task: task_txt,
    success: ok,
    score: v[:score],
    outcome: v,
    details: "#{v[:verdict]}(#{v[:score].to_f.round(2)}) #{v[:rationale]} | #{opts[:final].to_s[0, 200]}",
    session_id: session_id,
    tags: outcome_tags,
    judge_source: v[:source]
  )

  unless v[:training_score].nil?
    stages_run << :fold_judge
    fold_judge_into_metrics(session_id: session_id, score: v[:training_score], confidence: v[:confidence])
  end
  # R5 — close the live MDP episode with the ORM terminal reward.
  if defined?(PWN::AI::Agent::Policy) && Policy.respond_to?(:finish)
    stages_run << :policy
    Policy.finish(
      session_id: session_id,
      score: v[:training_score],
      attribution: v.dig(:verification, :runner_version) == 1 ? v.dig(:verification, :attribution) : nil,
      confidence: v[:confidence],
      verdict: v[:verdict],
      proxy_ok: ok,
      final: opts[:final],
      ts_state: opts[:ts_state]
    )
  end

  # R2 PRM — skip under hard cap (expensive LLM); keep under soft if heuristic path
  if over_hard.call || !defined?(Reward) || v[:training_score].nil? || !ok
    stages_skipped << :prm
  else
    stages_run << :prm
    Reward.prm(request: opts[:request], session_id: session_id)
  end

  # C3 HER — only on failure; skip hard
  if !ok && !v[:training_score].nil? && defined?(Curriculum) && !over_hard.call
    stages_run << :hindsight
    Curriculum.hindsight(request: opts[:request], final: opts[:final], session_id: session_id)
  else
    stages_skipped << :hindsight unless ok
  end

  # W3 calibrate — cheap; always write so Metrics.calibration is never empty.
  predicted = opts[:predicted]
  predicted = recover_predicted_from_session(session_id: session_id) if predicted.nil?
  stages_run << :calibrate if defined?(Curriculum)

  # reflect on success — skip soft/hard (LLM + memory writes)
  # M4.1 — also reflect when the request/final is a process SOP
  # (code hygiene) even if judge score < 0.6, so rubocop/rake
  # lessons still land in PWN::Memory.
  process_sop = !v[:training_score].nil? && process_sop_text?(text: "#{opts[:request]} #{opts[:final]}")
  if (ok || process_sop) && !over_soft.call
    stages_run << :reflect
    reflect(session_id: session_id)
  elsif ok || process_sop
    stages_skipped << :reflect
    # Cheap path under soft budget: still promote a canned process lesson
    if process_sop && defined?(PWN::Memory)
      promote_process_lesson(
        entry: {
          task: opts[:request].to_s[0, 120],
          success: ok,
          score: v[:score],
          details: opts[:final].to_s[0, 500],
          tags: %w[auto process_sop]
        }
      )
    end
  end

  # R3 sentinel — cheap disk math; always
  if defined?(Reward)
    stages_run << :sentinel
    Reward.sentinel
  end

  # E ambient extrospect — skip soft (can launch probes)
  if defined?(Extrospection) && !over_soft.call
    stages_run << :extrospect
    Extrospection.auto_extrospect(session_id: session_id)
  else
    stages_skipped << :extrospect
  end

  # Keep ~/.pwn RL stores lean on the feedback path (memory +
  # learning.jsonl + mistakes + sessions). Throttled; never raises.
  # Disk-only work: skip only hard budget / budget_hot.
  begin
    if !over_hard.call && !budget_hot && should_gc_stores?
      stages_run << :lean_gc
      gc_stores!(current_session_id: session_id)
    elsif should_gc_stores?
      stages_skipped << :lean_gc
    end
  rescue StandardError => e
    warn "[pwn-ai/learning] post-introspect lean swallowed: #{e.class}: #{e.message}"
  end

  {
    ok: ok,
    score: v[:score],
    elapsed_ms: elapsed_ms.call,
    budget_hot: budget_hot,
    stages_run: stages_run,
    stages_skipped: stages_skipped
  }
rescue StandardError => e
  warn "[pwn-ai/learning] auto_introspect swallowed: #{e.class}: #{e.message}"
  nil
end

.compact!(opts = {}) ⇒ Object

# File 'lib/pwn/ai/agent/learning.rb', line 1945

public_class_method def self.compact!(opts = {})
  max_baks = (opts[:max_baks] || learning_max_baks).to_i
  max_baks = 5 if max_baks <= 0
  dir = File.dirname(LEARNING_FILE)
  baks = Dir[File.join(dir, '*.bak*')].sort_by { |p| File.mtime(p) }.reverse
  pruned = 0
  baks.drop(max_baks).each do |path|
    File.delete(path)
    pruned += 1
  end
  { pruned: pruned, kept: [baks.length, max_baks].min, max_baks: max_baks }
end

.consistency_check(opts = {}) ⇒ Object

# File 'lib/pwn/ai/agent/learning.rb', line 174

public_class_method def self.consistency_check(opts = {})
  outcome = opts[:outcome]
  return :ok unless outcome.is_a?(Hash)
  return :disputed if Reward.resolve_outcome(outcome: outcome)[:success] != opts[:success]

  :ok
end

.consolidate(opts = {}) ⇒ Object

Supported Method Parameters

removed = PWN::AI::Agent::Learning.consolidate( max_entries: 'optional - hard cap on PWN::Memory size (default MAX_MEMORY_ENTRIES)' )

Deduplicates near-identical lesson values and prunes the oldest entries once the cap is exceeded so the injected MEMORY block stays high-signal.

# File 'lib/pwn/ai/agent/learning.rb', line 849

public_class_method def self.consolidate(opts = {})
  cap = opts[:max_entries] || MAX_MEMORY_ENTRIES
  return { removed: 0 } unless defined?(PWN::Memory)

  mem = nil
  load_err = nil
  begin
    mem = PWN::Memory.load
  rescue StandardError => e
    load_err = e
  end
  if load_err
    warn "[pwn-ai/learning] consolidate aborted (memory load failed): #{load_err.class}: #{load_err.message}"
    return { removed: 0, aborted: true, error: "#{load_err.class}: #{load_err.message}" }
  end
  removed = []

  # M1 — semantic clustering: embed :lesson entries, greedy-merge
  # near-duplicates (cosine ≥ 0.92) via Reflect into ONE imperative
  # lesson. Falls back to sha-dedup when no embed backend.
  removed.concat(semantic_merge(mem: mem)) if defined?(PWN::MemoryIndex) && PWN::MemoryIndex.available?

  seen = {}
  mem.each do |k, v|
    sig = Digest::SHA256.hexdigest(v[:value].to_s.strip.downcase)[0, 16]
    seen[sig] ? removed << k : seen[sig] = k
  end
  removed.uniq.each { |k| mem.delete(k) }

  # M3 — evict by (age/ttl) / (importance × confidence), NOT
  # oldest-first. Hand-written high-value lessons survive; low-
  # confidence :heuristic auto-gen self-evicts first.
  if mem.size > cap
    now = Time.now.utc
    scored = mem.map do |k, v|
      if defined?(PWN::Memory) && PWN::Memory.respond_to?(:protected_entry?) &&
         PWN::Memory.protected_entry?(key: k, entry: v)
        next [k, Float::INFINITY]
      end

      age_d = (now - Time.parse(v[:timestamp].to_s)) / 86_400.0
      ttl_d = (v[:ttl].to_f / 86_400.0)
      imp   = (v[:importance] || 0.5).to_f.clamp(0.05, 1.0)
      conf  = (v[:confidence] || (v[:source].to_s == 'human' ? 0.95 : 0.5)).to_f.clamp(0.05, 1.0)
      staleness = ttl_d.positive? ? age_d / ttl_d : age_d / 90.0
      # lower score = drop first; Infinity protected sorts last
      [k, -(staleness / (imp * conf))]
    rescue StandardError
      [k, 0.0]
    end
    # sort ascending by score so lowest (most stale/low-imp) first
    ordered = scored.sort_by { |_k, s| s }
    drop = []
    ordered.each do |pair|
      k = pair[0]
      break if mem.size - drop.size <= cap
      next if defined?(PWN::Memory) && PWN::Memory.respond_to?(:protected_entry?) &&
              PWN::Memory.protected_entry?(key: k, entry: mem[k])

      drop << k
    end
    drop.each { |k| mem.delete(k) }
    removed.concat(drop)
  end
  PWN::Memory.save(mem: mem, force: mem.empty?)
  if PWN::Memory.respond_to?(:lean!)
    begin
      PWN::Memory.lean!
    rescue StandardError => e
      warn "[pwn-ai/learning] post-consolidate memory.lean! swallowed: #{e.class}: #{e.message}"
    end
  end
  { removed: removed.uniq.length, remaining: mem.size }
end

.disputed_save(opts = {}) ⇒ Object

# File 'lib/pwn/ai/agent/learning.rb', line 182

public_class_method def self.disputed_save(opts = {})
  entry = opts[:entry] || {}
  FileUtils.mkdir_p(File.dirname(DISPUTED_FILE))
  File.open(DISPUTED_FILE, 'a') { |f| f.puts(JSON.generate(entry)) }
  entry
end

.distill_skill(opts = {}) ⇒ Object

Supported Method Parameters

skill = PWN::AI::Agent::Learning.distill_skill( name: 'required - snake_case name for the new skill', session_id: 'optional - PWN::Sessions id to mine (uses its transcript)', content: 'optional - explicit markdown body; overrides transcript mining', references: 'optional - Array of reference URLs / CWE / CVE / ATT&CK ids' )

# File 'lib/pwn/ai/agent/learning.rb', line 468

public_class_method def self.distill_skill(opts = {})
  raise 'ERROR: name is required' if opts[:name].to_s.strip.empty?

  body = opts[:content].to_s
  body = build_skill_from_session(session_id: opts[:session_id], name: opts[:name]) if body.strip.empty? && opts[:session_id]
  raise 'ERROR: content or session_id is required' if body.strip.empty?

  root = skills_dir
  out  = PWN::Config.write_skill(
    name: opts[:name],
    description: opts[:description],
    content: body,
    references: opts[:references],
    pwn_skills_path: root
  )
  PWN::Config.load_skills(pwn_skills_path: root) if PWN::Config.respond_to?(:load_skills)
  note_outcome(task: "distill_skill:#{out[:name]}", success: true, details: "Saved #{out[:path]}", tags: %w[skill auto])
  out.merge(saved: true)
end

.exemplars_for(opts = {}) ⇒ Object

Supported Method Parameters

msgs = PWN::AI::Agent::Learning.exemplars_for( request: 'required - current user request', limit: 'optional - max exemplar traces to return (default 1)', max_msgs: 'optional - cap on messages per exemplar (default 6)' )

Retrieval-augmented BEHAVIOUR: keyword-matches request against prior successful outcomes in learning.jsonl, loads the matching session, and compresses its (user, tool, assistant) trace into a short few-shot exemplar Loop.run splices between system and user. Local models are dramatically better with 1 concrete example than with 25 abstract lessons.

# File 'lib/pwn/ai/agent/learning.rb', line 335

public_class_method def self.exemplars_for(opts = {})
  request  = opts[:request].to_s.downcase
  limit    = (opts[:limit]    || 1).to_i
  max_msgs = (opts[:max_msgs] || 6).to_i
  return [] if request.strip.empty?

  tokens = request.scan(/[a-z0-9_]{3,}/).uniq
  return [] if tokens.empty?

  now = Time.now.utc
  # C2 — prioritized replay: priority = judge_score × recency_decay × keyword_sim
  # C2 — strict success:true only (excludes HER success:'soft'). Also
  # down-weight any residual hindsight-tagged rows so partial failures
  # never launder into full-strength few-shot exemplars.
  # P20 — strict success:true AND prefer high judge scores. Drop rows
  # with explicit low ORM score so proxy-true / judge-low cannot be few-shot.
  pool = outcomes(limit: 500, success: true).reject { |r| r[:session_id].to_s.empty? }
  pool = pool.reject { |r| r.key?(:score) && r[:score].to_f < 0.6 }
  scored = pool.map do |r|
    sim   = tokens.count { |t| r[:task].to_s.downcase.include?(t) }.to_f / tokens.length
    age_d = (now - Time.parse(r[:timestamp].to_s)) / 86_400.0
    decay = Math.exp(-age_d / 30.0)
    score = (r[:score] || 1.0).to_f
    tags  = Array(r[:tags]).map(&:to_s)
    # HER / soft / hindsight → 0.35× so they cannot dominate C2 priority
    score *= 0.35 if r[:success].to_s == 'soft' || tags.intersect?(%w[hindsight her soft])
    [r, sim * decay * score]
  rescue StandardError
    [r, 0.0]
  end
  hits = scored.reject { |_, pr| pr <= 0.0 }.sort_by { |_, pr| -pr }.first(limit).map(&:first)

  hits.flat_map { |r| compress_exemplar(session_id: r[:session_id], max_msgs: max_msgs) }
rescue StandardError
  []
end

.export_finetune(opts = {}) ⇒ Object

# File 'lib/pwn/ai/agent/learning.rb', line 392

public_class_method def self.export_finetune(opts = {})
  fmt       = (opts[:format] || :sharegpt).to_sym
  min_tools = (opts[:min_tools] || 1).to_i
  min_score = (opts[:min_score] || SFT_MIN_SCORE).to_f
  compress  = opts.key?(:compress) ? opts[:compress] : true
  FileUtils.mkdir_p(FINETUNE_DIR)
  out = opts[:out] || File.join(FINETUNE_DIR, "pwn-#{Time.now.utc.strftime('%Y%m%d')}.jsonl")

  # 4.1 / P12 — exclude HER soft-success + low-score + untagged auto flail
  gold = outcomes(limit: 10_000, success: true).reject do |r|
    tags = Array(r[:tags]).map(&:to_s)
    soft = r[:success].to_s == 'soft' || tags.intersect?(%w[hindsight her soft])
    low  = !r[:score].nil? && r[:score].to_f < min_score
    # require a score when present in corpus era that has scores
    soft || low
  end
  # prefer highest-score outcome per session
  by_sid = {}
  gold.each do |r|
    sid = r[:session_id].to_s
    next if sid.empty?

    prev = by_sid[sid]
    by_sid[sid] = r if prev.nil? || r[:score].to_f >= prev[:score].to_f
  end
  sids = by_sid.keys
  rows = 0
  dropped = { tools: 0, empty: 0, load: 0 }
  File.open(out, 'w') do |f|
    sids.each do |sid|
      t = begin
        PWN::Sessions.load(session_id: sid)
      rescue StandardError
        dropped[:load] += 1
        next
      end
      tool_n = t.count { |e| e[:role].to_s == 'tool' }
      if tool_n < min_tools
        dropped[:tools] += 1
        next
      end

      conv = if compress
               compress_finetune_trace(transcript: t, max_tool_chars: SFT_MAX_TOOL_CHARS)
             else
               t.map { |e| { role: e[:role].to_s, content: e[:content].to_s } }
                .reject { |e| e[:role] == 'system' && e[:content].start_with?('Session started') }
             end
      if conv.nil? || conv.empty? || conv.none? { |m| m[:role].to_s == 'assistant' }
        dropped[:empty] += 1
        next
      end

      line = case fmt
             when :openai_jsonl then { messages: conv }
             else { conversations: conv.map { |m| { from: sharegpt_role(role: m[:role]), value: m[:content] } } }
             end
      f.puts(JSON.generate(line))
      rows += 1
    end
  end
  {
    path: out, format: fmt, sessions: sids.length, samples: rows,
    bytes: File.size(out), min_score: min_score, compressed: compress,
    dropped: dropped
  }
end

.flip_last_outcome(opts = {}) ⇒ Object

# File 'lib/pwn/ai/agent/learning.rb', line 811

public_class_method def self.flip_last_outcome(opts = {})
  return { flipped: false } unless File.exist?(LEARNING_FILE)

  lines = File.readlines(LEARNING_FILE)
  return { flipped: false } if lines.empty?

  last = JSON.parse(lines.last, symbolize_names: true)
  return { flipped: false } if opts[:session_id] && last[:session_id] && last[:session_id] != opts[:session_id]
  return { flipped: false } unless last[:success]

  last[:success]    = false
  last[:flipped_by] = 'user_correction'
  last[:details]    = "#{last[:details]} | CORRECTED: #{opts[:reason].to_s[0, 200]}".strip
  last[:score]      = 0.0
  lines[-1] = "#{JSON.generate(last)}\n"
  File.write(LEARNING_FILE, lines.join)
  if defined?(Reward) && Reward.respond_to?(:record_preference)
    Reward.record_preference(
      prompt: last[:task].to_s,
      rejected: last[:details].to_s,
      chosen: opts[:reason].to_s,
      source: :user_correction
    )
  end
  { flipped: true, id: last[:id], rejected: last[:details].to_s[0, 2_000] }
rescue StandardError
  { flipped: false }
end

.gc_stores!(opts = {}) ⇒ Object

One-shot lean across memory + learning + mistakes + sessions. Called from auto_introspect (throttled) so the RL feedback loop keeps ~/.pwn high-signal without a manual learning_gc_stores turn.

Supported Method Parameters

result = PWN::AI::Agent::Learning.gc_stores!( dry_run: 'optional - Boolean (default false)', current_session_id: 'optional - never delete this sessions id', max_entries: 'optional - Memory consolidate cap', max_rows: 'optional - learning.jsonl cap', retain_days: 'optional - outcome / session age floor' )

# File 'lib/pwn/ai/agent/learning.rb', line 1725

public_class_method def self.gc_stores!(opts = {})
  dry = opts[:dry_run] ? true : false
  res = lean!(
    dry_run: dry,
    max_entries: opts[:max_entries],
    max_rows: opts[:max_rows],
    retain_days: opts[:retain_days],
    recent_days: opts[:recent_days],
    details_max: opts[:details_max],
    gold_min_score: opts[:gold_min_score]
  )
  res[:mistakes] = if defined?(Mistakes) && Mistakes.respond_to?(:lean!)
                     Mistakes.lean!(dry_run: dry)
                   else
                     { skipped: true }
                   end
  res[:policy] = if defined?(PWN::AI::Agent::Policy) && Policy.respond_to?(:lean!)
                   Policy.lean!(dry_run: dry)
                 else
                   { skipped: true }
                 end
  res[:sessions] = if defined?(PWN::Sessions) && PWN::Sessions.respond_to?(:lean!)
                     sess_opts = { dry_run: dry }
                     sid = opts[:current_session_id].to_s
                     sess_opts[:current_session_id] = sid unless sid.empty?
                     sess_opts[:retain_days] = opts[:retain_days] if opts.key?(:retain_days)
                     sess_opts[:max_files] = opts[:max_files] if opts.key?(:max_files)
                     PWN::Sessions.lean!(**sess_opts)
                   else
                     { skipped: true }
                   end
  res
end

.help ⇒ Object

Display Usage for this Module

# File 'lib/pwn/ai/agent/learning.rb', line 1966

public_class_method def self.help
  puts "USAGE:
    # Run note outcome and return its result
    #{self}.note_outcome(
      task: 'required - short description of what was attempted',
      success: 'required - Boolean, did the attempt achieve its goal',
      details: 'optional - free-form notes / error / evidence',
      rationale: 'optional - judge explanation retained for consistency checks',
      session_id: 'optional - PWN::Sessions id this outcome belongs to',
      tags: 'optional - Array of String labels for later retrieval',
      score: 'optional - score value consumed by #note_outcome',
      outcome: 'optional - canonical Reward outcome; preserves evidence, verdict and training eligibility',
      judge_source: 'required - judge source value consumed by #note_outcome',
      predicted: 'optional - predicted value consumed by #note_outcome',
      confidence: 'optional - confidence value consumed by #note_outcome',
      engine: 'optional - engine value consumed by #note_outcome',
      verifier_verdict: 'optional - legacy diagnostic flag; cannot prove completion',
      verdict_class: 'optional - missing_artifact|wrong_path|unverified_claim|scope_miss|partial_coverage|style_only',
      remediation_hint: 'optional - one-line fix hint'
    )

    # List outcomes tagged conflicted (verifier PASS vs low judge).
    #{self}.list_conflicted(
      limit: 'optional - max entries (defaults to 50)'
    )

    # Rejudge conflicted outcomes from the original session; never boost scores blindly.
    #{self}.requeue_conflicted(
      dry_run: 'optional - true to count without writing'
    )

    # Prune excess *.bak siblings under the learning directory.
    #{self}.compact!(
      max_baks: 'optional - newest bak files to keep (defaults to 5)'
    )

    # Run outcomes and return its result
    #{self}.outcomes(
      limit: 'optional - max entries returned newest-first (default 50)',
      success: 'optional - filter by Boolean outcome',
      tag: 'optional - filter by tag substring'
    )

    # Run stats and return its result
    #{self}.stats

    # Run to context and return its result
    #{self}.to_context(
      limit: 'optional - number of recent outcomes to surface (default 5)'
    )

    # Run exemplars for and return its result
    #{self}.exemplars_for(
      request: 'required - current user request',
      limit: 'optional - max exemplar traces to return (default 1)',
      max_msgs: 'optional - cap on messages per exemplar (default 6)'
    )

    # Run export finetune and return its result
    #{self}.export_finetune(
      format: 'optional - format value consumed by #export_finetune (defaults to :sharegpt))',
      min_tools: 'optional - min tools value consumed by #export_finetune',
      min_score: 'optional - min score value consumed by #export_finetune',
      compress: 'optional - compress value consumed by #export_finetune',
      out: 'optional - out value consumed by #export_finetune'
    )

    # Run distill skill and return its result
    #{self}.distill_skill(
      name: 'required - snake_case name for the new skill',
      session_id: 'optional - PWN::Sessions id to mine (uses its transcript)',
      content: 'optional - explicit markdown body; overrides transcript mining',
      references: 'optional - Array of reference URLs / CWE / CVE / ATT&CK ids',
      description: 'optional - description value consumed by #distill_skill'
    )

    # Fold RL artefacts (mistakes / structured_fix / an explicit lesson)
    #{self}.update_skill(
      dry_run: 'optional - dry run value consumed by #update_skill',
      lesson: 'optional - lesson value consumed by #update_skill',
      signature: 'optional - signature value consumed by #update_skill',
      request: 'optional - request value consumed by #update_skill (defaults to opts[:query])',
      query: 'optional - search query string',
      name: 'required - binary or identifier name'
    )

    # Run reflect and return its result
    #{self}.reflect(
      session_id: 'required - PWN::Sessions id to analyse',
      dry_run: 'optional - when true, do not write to Memory/Skills (default false)'
    )

    # Called by Loop.run when PWN::Env[:ai][:agent][:auto_introspect] is
    #{self}.auto_introspect(
      session_id: 'required - id of the just-completed session',
      request: 'optional - original user request (for outcome logging)',
      final: 'optional - final assistant answer (for outcome logging)',
      inline: 'optional - inline value consumed by #auto_introspect',
      predicted: 'optional - predicted value consumed by #auto_introspect',
      plan: 'optional - plan value consumed by #auto_introspect',
      ts_state: 'optional - ts state value consumed by #auto_introspect'
    )

    # Run flip last outcome and return its result
    #{self}.flip_last_outcome(
      session_id: 'optional - session id value consumed by #flip_last_outcome',
      reason: 'optional - reason value consumed by #flip_last_outcome'
    )

    # Run consolidate and return its result
    #{self}.consolidate(
      max_entries: 'optional - hard cap on PWN::Memory size (default MAX_MEMORY_ENTRIES)'
    )

    # Run reset and return its result
    #{self}.reset

    # One-shot / on-load repair: rewrite tags+details where verdict label
    #{self}.reconcile_verdict_tags!(
      dry_run: 'optional - dry run value consumed by #reconcile_verdict_tags!'
    )

    # Run prune outcomes and return its result
    #{self}.prune_outcomes!(
      dry_run: 'optional - Boolean (default false)',
      max_rows: 'optional - hard cap (default MAX_OUTCOME_ROWS)',
      retain_days: 'optional - age floor for low-value drop',
      recent_days: 'optional - always keep newer than this',
      details_max: 'optional - details max value consumed by #prune_outcomes!',
      gold_min_score: 'optional - gold min score value consumed by #prune_outcomes!'
    )

    # Memory lean + outcome prune
    #{self}.lean!(
      dry_run: 'optional - dry run value consumed by #lean!',
      max_entries: 'optional - max entries value consumed by #lean!',
      max_rows: 'optional - max rows value consumed by #lean!',
      retain_days: 'optional - retain days value consumed by #lean!',
      recent_days: 'optional - recent days value consumed by #lean!',
      details_max: 'optional - details max value consumed by #lean!',
      gold_min_score: 'optional - gold min score value consumed by #lean!'
    )

    # One-shot lean across memory + learning + mistakes + sessions
    #{self}.gc_stores!(
      dry_run: 'optional - Boolean (default false)',
      current_session_id: 'optional - never delete this sessions id',
      max_entries: 'optional - Memory consolidate cap',
      max_rows: 'optional - learning.jsonl cap',
      retain_days: 'optional - outcome / session age floor',
      recent_days: 'optional - recent days value consumed by #gc_stores!',
      details_max: 'optional - details max value consumed by #gc_stores!',
      gold_min_score: 'optional - gold min score value consumed by #gc_stores!',
      max_files: 'optional - max files value consumed by #gc_stores!'
    )

    # One-shot GC of the pre-R1 garbage: drops every PWN::Memory entry
    #{self}.purge_noise

    # Record a lesson as candidate (injected with [UNVERIFIED] until verified).
    #{self}.lesson_record(
      text: 'required - lesson text to quarantine'
    )

    # Count a success or contradiction against a lesson id.
    #{self}.lesson_observe(
      id: 'required - lesson id from #lesson_record',
      success: 'required - true to count a success, false for a contradiction'
    )

    # Prompt block of non-demoted lessons; candidates prefixed [UNVERIFIED].
    #{self}.lesson_prompt(
      include_demoted: 'optional - include demoted lessons (defaults to false)'
    )

    # Compare structured decisions, never infer verification from prose.
    #{self}.consistency_check(
      outcome: 'optional - canonical outcome to compare against success',
      success: 'required - boolean success flag'
    )

    # Append a disputed outcome to ~/.pwn/learning/disputed.jsonl.
    #{self}.disputed_save(
      entry: 'required - Hash of the disputed learning row'
    )

    # Print the AUTHOR(S) string for this module.
    #{self}.authors
  "
  constants.sort
end

.lean!(opts = {}) ⇒ Object

Memory lean + outcome prune.

# File 'lib/pwn/ai/agent/learning.rb', line 1694

public_class_method def self.lean!(opts = {})
  dry = opts[:dry_run] ? true : false
  out = { dry_run: dry }
  out[:memory] = if defined?(PWN::Memory) && PWN::Memory.respond_to?(:lean!)
                   PWN::Memory.lean!(dry_run: dry)
                 else
                   { skipped: true }
                 end
  out[:memory_consolidate] = consolidate(max_entries: opts[:max_entries] || MAX_MEMORY_ENTRIES) unless dry
  out[:learning] = prune_outcomes!(
    dry_run: dry,
    max_rows: opts[:max_rows],
    retain_days: opts[:retain_days],
    recent_days: opts[:recent_days],
    details_max: opts[:details_max],
    gold_min_score: opts[:gold_min_score]
  )
  out
end

.lesson_observe(opts = {}) ⇒ Object

# File 'lib/pwn/ai/agent/learning.rb', line 1848

public_class_method def self.lesson_observe(opts = {})
  id = opts[:id].to_s
  store = lesson_store
  row = store[id]
  return nil unless row

  if opts[:success]
    row[:successes] = row[:successes].to_i + 1
    row[:state] = 'verified' if row[:successes] >= 2
  else
    row[:contradictions] = row[:contradictions].to_i + 1
    row[:state] = 'demoted' if row[:contradictions] >= 2
  end
  store[id] = row
  lesson_save(store: store)
  row
end

.lesson_prompt(opts = {}) ⇒ Object

# File 'lib/pwn/ai/agent/learning.rb', line 1866

public_class_method def self.lesson_prompt(opts = {})
  include_demoted = opts[:include_demoted] ? true : false
  lesson_store.values.filter_map do |row|
    next if !include_demoted && row[:state].to_s == 'demoted'

    tag = row[:state].to_s == 'verified' ? '' : '[UNVERIFIED] '
    "#{tag}#{row[:text]}"
  end.join("\n")
end

.lesson_record(opts = {}) ⇒ Object

# File 'lib/pwn/ai/agent/learning.rb', line 1837

public_class_method def self.lesson_record(opts = {})
  text = opts[:text].to_s
  raise 'ERROR: text is required' if text.empty?

  store = lesson_store
  id = Digest::SHA256.hexdigest(text)[0, 12]
  store[id] ||= { id: id, text: text, state: 'candidate', successes: 0, contradictions: 0 }
  lesson_save(store: store)
  store[id]
end

.list_conflicted(opts = {}) ⇒ Object

# File 'lib/pwn/ai/agent/learning.rb', line 1903

public_class_method def self.list_conflicted(opts = {})
  limit = (opts[:limit] || 50).to_i
  outcomes(limit: 500).select { |r| r[:status].to_s == 'conflicted' }.first(limit)
end

.note_outcome(opts = {}) ⇒ Object

Supported Method Parameters

entry = PWN::AI::Agent::Learning.note_outcome( task: 'required - short description of what was attempted', success: 'required - Boolean, did the attempt achieve its goal', details: 'optional - free-form notes / error / evidence', session_id: 'optional - PWN::Sessions id this outcome belongs to', tags: 'optional - Array of String labels for later retrieval' )

# File 'lib/pwn/ai/agent/learning.rb', line 93

public_class_method def self.note_outcome(opts = {})
  task    = opts[:task].to_s
  # 4.1 — allow success: 'soft' (HER) distinct from true/false
  raw_ok  = opts[:success]
  success = if ['soft', :soft].include?(raw_ok)
              'soft'
            else
              raw_ok ? true : false
            end
  raise 'ERROR: task is required' if task.strip.empty?

  tags = Array(opts[:tags]).map(&:to_s)
  details = opts[:details].to_s[0, OUTCOME_DETAILS_MAX]
  decision = if opts[:outcome].is_a?(Hash)
               Reward.resolve_outcome(outcome: opts[:outcome])
             elsif opts.key?(:score) && success != 'soft'
               Reward.resolve_outcome(outcome: { score: opts[:score], source: opts[:judge_source] || :manual, confidence: opts[:confidence] })
             end
  if decision || opts.key?(:score)
    score = decision ? decision[:score] : opts[:score].to_f
    want = (decision ? decision[:verdict] : verdict_for_score(score: score)).to_s
    tags = (tags - %w[solved partial wrong unknown]) << want
    details = details.sub(
      /\A(solved|partial|wrong|unknown)\(\d+(?:\.\d+)?\)/i,
      "#{want}(#{score.nil? ? 'unknown' : format('%.2f', score)})"
    )
    success = decision[:success] if decision
  end

  entry = {
    id: Digest::SHA256.hexdigest("#{task}-#{Time.now.to_f}")[0, 12],
    task: task,
    success: success,
    details: details,
    session_id: opts[:session_id],
    tags: tags,
    timestamp: Time.now.utc.iso8601
  }
  entry[:score] = score if decision || opts.key?(:score)
  if decision
    i[verdict confidence training_score decision_version verification verifier_verdict grounded critic_pass judge_score quality_score rationale].each do |key|
      entry[key] = decision[key] if decision.key?(key)
    end
    entry[:status] = 'unverified' if decision[:training_score].nil?
  end
  src = opts[:judge_source].to_s
  src = decision[:source].to_s if decision && decision[:source]
  entry[:judge_source] = src unless src.empty?
  vv = (decision ? decision[:verifier_verdict] : opts[:verifier_verdict] || opts['verifier_verdict']).to_s
  entry[:verifier_verdict] = vv unless vv.empty?
  vc = (opts[:verdict_class] || opts['verdict_class']).to_s
  entry[:verdict_class] = vc unless vc.empty?
  entry[:remediation_hint] = opts[:remediation_hint].to_s unless opts[:remediation_hint].to_s.empty?
  entry[:status] = 'conflicted' if opts[:verifier_verdict].to_s == 'pass' && !entry[:verification] && opts[:score].to_f < 0.6
  check = consistency_check(details: details, success: success, rationale: opts[:rationale])
  if check == :disputed
    entry[:status] = 'disputed'
    disputed_save(entry: entry)
    return entry
  end
  FileUtils.mkdir_p(File.dirname(LEARNING_FILE))
  File.open(LEARNING_FILE, 'a') { |f| f.puts(JSON.generate(entry)) }
  maybe_prune_outcomes!

  # M4 — default: outcomes live in learning.jsonl ONLY.
  # M4.1 — PROCESS SOPs (rubocop/rake/spec after code changes, etc.)
  # are promoted into PWN::Memory[:lesson] so PromptBuilder recall
  # survives across sessions. Without this, the agent re-learns
  # "run rubocop after every patch" every turn (empty memory.json).
  promote_process_lesson(entry: entry) if defined?(PWN::Memory) && !%w[conflicted unverified].include?(entry[:status].to_s)
  if opts.key?(:score) && (!decision || !decision[:training_score].nil?) && defined?(Curriculum) && Curriculum.respond_to?(:calibrate)
    pred = opts[:predicted]
    pred = Thread.current[:pwn_plan_predicted] if pred.nil?
    pred = opts[:confidence] if pred.nil?
    eng = opts[:engine]
    eng = (PWN::Env.dig(:ai, :active) if defined?(PWN::Env)) if eng.to_s.empty?
    Curriculum.calibrate(predicted: pred, actual: decision ? decision[:training_score] : opts[:score], engine: eng)
  end
  entry
end

.outcomes(opts = {}) ⇒ Object

Supported Method Parameters

rows = PWN::AI::Agent::Learning.outcomes( limit: 'optional - max entries returned newest-first (default 50)', success: 'optional - filter by Boolean outcome', tag: 'optional - filter by tag substring' )

# File 'lib/pwn/ai/agent/learning.rb', line 196

public_class_method def self.outcomes(opts = {})
  limit   = opts[:limit] || 50
  want_ok = opts.key?(:success) ? !opts[:success].nil? && opts[:success] != false : nil
  tag     = opts[:tag].to_s.downcase
  return [] unless File.exist?(LEARNING_FILE)

  rows = File.readlines(LEARNING_FILE).map do |l|
    JSON.parse(l, symbolize_names: true)
  rescue StandardError
    nil
  end
  rows.compact!
  rows.reject! { |r| r[:decision_version] && r[:training_score].nil? } unless want_ok.nil?
  rows.select! { |r| want_ok == true ? r[:success] == true : r[:success] == want_ok } unless want_ok.nil?
  rows.select! { |r| Array(r[:tags]).any? { |t| t.to_s.downcase.include?(tag) } } unless tag.empty?
  rows.reverse.first(limit)
end

.prune_outcomes!(opts = {}) ⇒ Object

Supported Method Parameters

result = PWN::AI::Agent::Learning.prune_outcomes!( dry_run: 'optional - Boolean (default false)', max_rows: 'optional - hard cap (default MAX_OUTCOME_ROWS)', retain_days: 'optional - age floor for low-value drop', recent_days: 'optional - always keep newer than this' )

Keep gold RL rows (success+score>=0.6+session_id), recent window, high-value tags, and near-miss failures. Dedupe by task+success keeping best score. Truncate details. Never sacrifices exemplar pool.

# File 'lib/pwn/ai/agent/learning.rb', line 1567

public_class_method def self.prune_outcomes!(opts = {})
  dry = opts[:dry_run] ? true : false
  max_rows = (opts[:max_rows] || MAX_OUTCOME_ROWS).to_i
  retain_days = (opts[:retain_days] || OUTCOME_RETAIN_DAYS).to_f
  recent_days = (opts[:recent_days] || OUTCOME_RECENT_DAYS).to_f
  details_max = (opts[:details_max] || OUTCOME_DETAILS_MAX).to_i
  gold_min = (opts[:gold_min_score] || GOLD_MIN_SCORE).to_f

  return { kept: 0, removed: 0, skipped: true } unless File.exist?(LEARNING_FILE)

  before_bytes = File.size(LEARNING_FILE)
  rows = File.readlines(LEARNING_FILE).map do |l|
    JSON.parse(l, symbolize_names: true)
  rescue StandardError
    nil
  end.compact

  now = Time.now.utc
  age_days = lambda do |r|
    (now - Time.parse(r[:timestamp].to_s)) / 86_400.0
  rescue StandardError
    999.0
  end

  protected_row = lambda do |r|
    tags = Array(r[:tags]).map(&:to_s)
    a = age_days.call(r)
    return true if a <= recent_days
    return true if r[:success] == true && r.key?(:score) && r[:score].to_f >= gold_min && r[:session_id].to_s != ''
    return true if r[:success] == true && !r.key?(:score) && r[:session_id].to_s != '' && a <= retain_days
    return true if tags.intersect?(HIGH_VALUE_TAGS)
    return true if r[:success] == false && r.key?(:score) && r[:score].to_f >= 0.5

    false
  end

  rows.reject! { |r| r[:task].to_s.strip.empty? }

  best = {}
  rows.each do |r|
    key = [r[:task].to_s.strip.downcase.gsub(/\s+/, ' ')[0, 160], r[:success].to_s]
    prev = best[key]
    if prev.nil?
      best[key] = r
    else
      ps = prev.key?(:score) ? prev[:score].to_f : -1.0
      rs = r.key?(:score) ? r[:score].to_f : -1.0
      better = rs > ps || (rs == ps && r[:timestamp].to_s > prev[:timestamp].to_s)
      better ||= protected_row.call(r) && !protected_row.call(prev)
      best[key] = r if better
    end
  end
  deduped = best.values
  removed_dupes = rows.size - deduped.size

  truncated = 0
  deduped.each do |r|
    d = r[:details].to_s
    next if d.bytesize <= details_max

    r[:details] = "#{d[0, details_max]}…[compacted]"
    truncated += 1
  end

  protected, unprotected = deduped.partition { |r| protected_row.call(r) }

  kept_unprot = unprotected.reject do |r|
    a = age_days.call(r)
    tags = Array(r[:tags]).map(&:to_s)
    score = r.key?(:score) ? r[:score].to_f : 1.0
    noise_tags = (tags - LOW_VALUE_ONLY_TAGS).empty? && tags.any?
    a > retain_days && noise_tags && score < 0.4
  end

  gold = (protected + kept_unprot).select do |r|
    r[:success] == true && r[:session_id].to_s != '' &&
      (!r.key?(:score) || r[:score].to_f >= gold_min)
  end
  if gold.size < EXEMPLARS_POOL_MIN
    need = EXEMPLARS_POOL_MIN - gold.size
    extra = unprotected.select { |r| r[:success] == true && r[:session_id].to_s != '' }
                       .sort_by { |r| r[:timestamp].to_s }
                       .last(need)
    kept_unprot = (kept_unprot + extra).uniq
  end

  fails = (protected + kept_unprot).reject { |r| r[:success] == true }
  if fails.size < FAILURE_WINDOW_MIN
    need = FAILURE_WINDOW_MIN - fails.size
    extra = unprotected.reject { |r| r[:success] == true }
                       .sort_by { |r| r[:timestamp].to_s }
                       .last(need)
    kept_unprot = (kept_unprot + extra).uniq
  end

  kept = (protected + kept_unprot).uniq
  if kept.size > max_rows
    prot_ids = protected.map { |r| r[:id] }.compact
    over = kept.size - max_rows
    victims = kept.reject { |r| prot_ids.include?(r[:id]) }
                  .sort_by { |r| r[:timestamp].to_s }
                  .first(over)
    v_ids = victims.map { |r| r[:id] }
    kept = kept.reject { |r| v_ids.include?(r[:id]) }
  end

  kept = kept.sort_by { |r| r[:timestamp].to_s }

  atomic_jsonl_write(path: LEARNING_FILE, rows: kept) unless dry

  {
    kept: kept.size,
    removed: (rows.size - kept.size) + removed_dupes,
    deduped: removed_dupes,
    truncated_details: truncated,
    protected: protected.size,
    bytes_before: before_bytes,
    bytes_after: if dry
                   before_bytes
                 else
                   (File.exist?(LEARNING_FILE) ? File.size(LEARNING_FILE) : 0)
                 end,
    dry_run: dry
  }
end

.purge_noise ⇒ Object

Supported Method Parameters

PWN::AI::Agent::Learning.purge_noise

One-shot GC of the pre-R1 garbage: drops every PWN::Memory entry matching the old SUCCESS: <req> — <final> / Avoid repeating failure pattern from <tool>: {"success":true shapes. Run once after upgrading; subsequent writes never produce these.

# File 'lib/pwn/ai/agent/learning.rb', line 1811

public_class_method def self.purge_noise
  return { removed: 0 } unless defined?(PWN::Memory)

  mem = nil
  load_err = nil
  begin
    mem = PWN::Memory.load
  rescue StandardError => e
    load_err = e
  end
  if load_err
    warn "[pwn-ai/learning] purge_noise aborted (memory load failed): #{load_err.class}: #{load_err.message}"
    return { removed: 0, aborted: true, error: "#{load_err.class}: #{load_err.message}" }
  end
  before = mem.size
  mem.reject! do |_k, v|
    next false unless v[:category].to_s == 'lesson'

    val = v[:value].to_s
    val.start_with?('SUCCESS: ', 'FAILURE: ') ||
      val.match?(/\AAvoid repeating failure pattern from \w+: .{0,5}\{"success":true/)
  end
  PWN::Memory.save(mem: mem, force: mem.empty?)
  { removed: before - mem.size, remaining: mem.size }
end

.reconcile_verdict_tags!(opts = {}) ⇒ Object

One-shot / on-load repair: rewrite tags+details where verdict label disagrees with score (solved @ 0.3 etc.). Safe to call repeatedly.

# File 'lib/pwn/ai/agent/learning.rb', line 1136

public_class_method def self.reconcile_verdict_tags!(opts = {})
  return { repaired: 0 } unless File.exist?(LEARNING_FILE)

  dry = opts[:dry_run] ? true : false
  repaired = 0
  lines = File.readlines(LEARNING_FILE)
  out = lines.map do |l|
    r = JSON.parse(l, symbolize_names: true)
    next l if r[:decision_version]

    score = r.key?(:score) ? r[:score].to_f : nil
    next l if score.nil?

    want = verdict_for_score(score: score).to_s
    tags = Array(r[:tags]).map(&:to_s)
    stale = tags & %w[solved partial wrong unknown]
    next l if stale.empty? || stale.include?(want)

    repaired += 1
    next l if dry

    cleaned = tags - %w[solved partial wrong unknown]
    cleaned << want
    r[:tags] = cleaned
    # Fix leading "solved(0.3)" style details head when present
    det = r[:details].to_s
    r[:details] = det.sub(
      /\A(solved|partial|wrong|unknown)\(\d+(?:\.\d+)?\)/i,
      "#{want}(#{format('%.2f', score)})"
    )
    r[:success] = (score >= 0.6) if [true, false].include?(r[:success])
    "#{JSON.generate(r)}\n"
  rescue StandardError
    l
  end
  File.write(LEARNING_FILE, out.join) if !dry && repaired.positive?
  { repaired: repaired, dry_run: dry }
rescue StandardError => e
  { repaired: 0, error: "#{e.class}: #{e.message}" }
end

.reflect(opts = {}) ⇒ Object

Supported Method Parameters

report = PWN::AI::Agent::Learning.reflect( session_id: 'required - PWN::Sessions id to analyse', dry_run: 'optional - when true, do not write to Memory/Skills (default false)' )

Uses PWN::AI::Agent::Reflect (when available) to LLM-summarise the session into structured lessons. Falls back to a heuristic extractor when module_reflection is disabled so learning never stops.

# File 'lib/pwn/ai/agent/learning.rb', line 557

public_class_method def self.reflect(opts = {})
  session_id = opts[:session_id]
  dry_run    = opts[:dry_run] ? true : false
  raise 'ERROR: session_id is required' if session_id.to_s.empty?

  transcript = PWN::Sessions.load(session_id: session_id)
  return { session_id: session_id, lessons: [], reason: 'empty transcript' } if transcript.empty?

  lessons = introspective_lessons(transcript: transcript)
  source, conf = lessons.empty? ? [:heuristic, 0.3] : [:reflect, 0.8]
  lessons = heuristic_lessons(transcript: transcript) if lessons.empty?

  saved = []
  lessons.each do |l|
    next if l.to_s.strip.empty?

    key = :"reflect_#{session_id}_#{Digest::SHA256.hexdigest(l)[0, 8]}"
    # M3 — provenance + confidence + ttl so consolidate evicts
    # low-confidence heuristic lessons before hand-written ones.
    PWN::Memory.remember(key: key, value: l, category: :lesson, source: source, confidence: conf, importance: conf, ttl: source == :heuristic ? 7 * 86_400 : nil) unless dry_run
    saved << { key: key, lesson: l }
  end
  consolidate unless dry_run

  { session_id: session_id, lessons: saved, count: saved.length, dry_run: dry_run }
end

.requeue_conflicted(opts = {}) ⇒ Object

# File 'lib/pwn/ai/agent/learning.rb', line 1908

public_class_method def self.requeue_conflicted(opts = {})
  dry = opts[:dry_run] ? true : false
  rows = list_conflicted(limit: 10_000)
  return { rescored: 0, dry_run: dry } if rows.empty? || dry

  n = 0
  rows.each do |r|
    next if r[:session_id].to_s.empty?

    transcript = PWN::Sessions.load(session_id: r[:session_id])
    user_idx = transcript.rindex { |entry| entry[:role].to_s == 'user' }
    next unless user_idx

    request = transcript[user_idx][:content].to_s
    final = transcript[(user_idx + 1)..].reverse.find { |entry| entry[:role].to_s == 'assistant' }
    next unless final && (request == r[:task].to_s || display_task(task: request) == r[:task].to_s)

    outcome = Reward.judge(request: request, final: final[:content], session_id: r[:session_id], commit: false)
    fresh = note_outcome(
      task: request,
      session_id: r[:session_id],
      outcome: outcome,
      details: outcome[:rationale].to_s,
      tags: %w[requeue],
      judge_source: outcome[:source]
    )
    updated = File.readlines(LEARNING_FILE).map do |line|
      row = JSON.parse(line, symbolize_names: true)
      row.merge!(status: 'rejudged', rescore_id: fresh[:id]) if row[:id] == r[:id]
      "#{JSON.generate(row)}\n"
    end
    File.write(LEARNING_FILE, updated.join)
    n += 1
  end
  { rescored: n, dry_run: false }
end

.reset ⇒ Object

Supported Method Parameters

PWN::AI::Agent::Learning.reset

# File 'lib/pwn/ai/agent/learning.rb', line 927

public_class_method def self.reset
  FileUtils.rm_f(LEARNING_FILE)
  { cleared: true }
end

.stats ⇒ Object

Supported Method Parameters

stats = PWN::AI::Agent::Learning.stats

# File 'lib/pwn/ai/agent/learning.rb', line 217

public_class_method def self.stats
  rows   = outcomes(limit: 10_000)
  total  = rows.length
  rows = rows.reject { |r| (r[:decision_version] && r[:training_score].nil?) || r[:success].nil? }
  evaluated = rows.length
  ok     = rows.count { |r| r[:success] == true }
  skills = defined?(PWN::Skills) && PWN::Skills.is_a?(Hash) ? PWN::Skills.keys.length : 0
  mem    = defined?(PWN::Memory) ? PWN::Memory.load.keys.length : 0
  raw    = evaluated.positive? ? (ok.to_f / evaluated).round(3) : 0.0
  jmean  = evaluated.positive? ? weighted_judge_mean(rows: rows) : nil
  distrust = 0.0
  distrust = Reward.proxy_distrust.to_f.clamp(0.0, 1.0) if defined?(Reward) && Reward.respond_to?(:proxy_distrust)
  orm = rows.select { |r| r[:judge_source].to_s != 'heuristic' && r[:source].to_s != 'heuristic' }
  heur = rows.select { |r| r[:judge_source].to_s == 'heuristic' || r[:source].to_s == 'heuristic' }
  orm_n = orm.length
  heur_n = heur.length
  orm_ok = orm.count { |r| r[:success] == true }
  heur_ok = heur.count { |r| r[:success] == true }
  {
    total_outcomes: total,
    unknown_outcomes: total - evaluated,
    successes: ok,
    failures: rows.count { |r| r[:success] == false },
    success_rate: raw,
    success_rate_orm: orm_n.positive? ? (orm_ok.to_f / orm_n).round(3) : 0.0,
    success_rate_heur: heur_n.positive? ? (heur_ok.to_f / heur_n).round(3) : 0.0,
    orm_n: orm_n,
    heur_n: heur_n,
    adjusted_success_rate: discount_success_rate(proxy: raw, judge: jmean, distrust: distrust),
    proxy_distrust: distrust,
    skills_known: skills,
    memory_entries: mem,
    judge_mean: jmean,
    reward_sentinel: (Reward.sentinel if defined?(Reward)),
    calibration: (Metrics.calibration if defined?(Metrics) && Metrics.respond_to?(:calibration)),
    preference_pairs: (Reward.preferences(limit: 100_000).length if defined?(Reward)),
    tool_metrics: (Metrics.summary(limit: 5) if defined?(Metrics)),
    extrospection: (Extrospection.stats if defined?(Extrospection))
  }
end

.to_context(opts = {}) ⇒ Object

Supported Method Parameters

ctx = PWN::AI::Agent::Learning.to_context( limit: 'optional - number of recent outcomes to surface (default 5)' )

# File 'lib/pwn/ai/agent/learning.rb', line 263

public_class_method def self.to_context(opts = {})
  limit = opts[:limit] || 5
  # Fetch a wider window so prefer_primary_tasks can drop critic/red_team
  # envelope rows (REQUEST:/GOAL: prefixes) without starving the block.
  rows  = prefer_primary_tasks(rows: outcomes(limit: limit * 4)).first(limit)
  fails = prefer_primary_tasks(rows: outcomes(limit: 200, success: false))
  fails = fails.reject { |r| r[:status].to_s == 'conflicted' }
  fails = fails.reject { |r| r[:verifier_verdict].to_s == 'pass' }
  fails = fails.reject { |r| r[:details].to_s.match?(/\bPASS\b/) && r[:success] == true }
  fails = fails.select do |r|
    v = r[:verdict].to_s
    v == 'wrong' || v == 'refused' || (r[:score].to_f < 0.3 && !r[:details].to_s.match?(/\bPASS\b/)) || r[:success] == false
  end
  fails = fails.reject { |r| r[:success] == true }
  # Do not mirror the same ids under both headings — that doubled the
  # failure signal and made RECENT OUTCOMES == RECENT FAILURES when the
  # last N attempts all failed (the injected block looked "stuck").
  row_ids = rows.map { |r| r[:id] }.compact
  fails = fails.reject { |r| row_ids.include?(r[:id]) }.first(limit)
  return '' if rows.empty? && fails.empty?

  fmt = lambda do |r|
    unknown = r[:status].to_s == 'unverified' || (r[:decision_version] && r[:training_score].nil?)
    flag = case r[:success]
           when true then '✓'
           when 'soft', :soft then '∼'
           else '✗'
           end
    score = r.key?(:score) ? format('%.2f', r[:score].to_f) : '-'
    if unknown
      flag = '?'
      score = 'unknown'
    end
    task  = display_task(task: r[:task])
    line  = "  #{flag} [#{score}] #{task} (#{r[:timestamp]})"
    # Surface a one-line cause crumb so the agent can actually learn
    # from failures instead of only seeing that they failed.
    if r[:success] != true && !unknown
      if r[:verdict_class].to_s == ''
        crumb = cause_crumb(details: r[:details])
        line += "\n      cause: #{crumb}" unless crumb.empty?
      else
        line += "\n      cause: #{r[:verdict_class]} #{r[:remediation_hint]}"
      end
    end
    line
  end
  s   = stats
  jm  = s[:judge_mean]
  d   = s[:proxy_distrust].to_f
  rate = d > 0.05 ? s[:adjusted_success_rate] : s[:success_rate]
  tag  = d > 0.05 ? ' adj' : ''
  hdr = "RECENT OUTCOMES (success_rate=#{(rate.to_f * 100).round(1)}%#{tag} success=orm:#{(s[:success_rate_orm].to_f * 100).round(1)}%(#{s[:orm_n]}) / heur:#{(s[:success_rate_heur].to_f * 100).round(1)}%(#{s[:heur_n]})#{" judge_mean=#{jm}" if jm} over #{s[:total_outcomes]} attempts)"
  out = "#{hdr}\n#{rows.map(&fmt).join("\n")}\n"
  out += "RECENT FAILURES (learn from these — do not repeat)\n#{fails.map(&fmt).join("\n")}\n" unless fails.empty?
  "#{out}\n"
end

.update_skill(opts = {}) ⇒ Object

Fold RL artefacts (mistakes / structured_fix / an explicit lesson) into an existing skill. Does not create skills (use distill_skill / skill_create). Does not write loop-law. Dedupes by mistake signature.

# File 'lib/pwn/ai/agent/learning.rb', line 491

public_class_method def self.update_skill(opts = {})
  dry = opts[:dry_run] ? true : false
  lesson = opts[:lesson].to_s.strip
  notes = rl_skill_notes(
    signature: opts[:signature],
    request: opts[:request] || opts[:query],
    lesson: lesson
  )
  return { updated: false, reason: 'no rl notes' } if notes.empty?

  target = locate_skill_for_update(
    name: opts[:name],
    query: opts[:query] || opts[:request] || notes.map { |n| n[:text] }.join(' ')
  )
  return { updated: false, reason: 'no matching skill', notes: notes } unless target

  body = skill_body_without_frontmatter(meta: target[:meta])
  added = []
  notes.each do |note|
    next if body.include?(note[:id])

    added << note
  end
  return { updated: false, name: target[:name], reason: 'already folded', notes: notes } if added.empty?

  block = added.map { |n| "- [#{n[:id]}] #{n[:text]}" }.join("\n")
  body = if body.match?(/^\#{1,3}\s*RL feedback\s*$/i)
           "#{body.rstrip}\n#{block}\n"
         else
           "#{body.rstrip}\n\n## RL feedback\n#{block}\n"
         end
  return { updated: false, dry_run: true, name: target[:name], added: added.map { |n| n[:text] } } if dry

  root = skills_dir
  out = PWN::Config.write_skill(
    name: target[:name],
    description: target[:meta][:description],
    content: body,
    references: target[:meta][:references],
    license: (target[:meta][:frontmatter] || {})['license'],
    allowed_tools: target[:meta][:allowed_tools],
    metadata: (target[:meta][:frontmatter] || {})['metadata'],
    pwn_skills_path: root
  )
  PWN::Config.load_skills(pwn_skills_path: root) if PWN::Config.respond_to?(:load_skills)
  note_outcome(
    task: "skills_update:#{target[:name]}",
    success: true,
    details: "Folded #{added.length} RL note(s)",
    tags: %w[skill rl]
  )
  out.merge(updated: true, added: added.map { |n| n[:id] })
rescue StandardError => e
  { updated: false, error: "#{e.class}: #{e.message}" }
end