Class: Broadlistening::CsvLoader

Inherits:
Object
  • Object
show all
Defined in:
lib/broadlistening/csv_loader.rb

Overview

Loads comments from CSV files with Kouchou-AI compatible format.

This loader supports the CSV format used by Kouchou-AI (Python version), enabling compatibility testing between the two implementations.

Examples:

Loading a Kouchou-AI format CSV

comments = CsvLoader.load("inputs/example-polis.csv")

Loading with property columns

comments = CsvLoader.load("data.csv", property_names: ["agrees", "disagrees"])

Using custom column mapping

comments = CsvLoader.load("custom.csv", column_mapping: {
  id: "my_id_column",
  body: "my_body_column"
})

Constant Summary collapse

KOUCHOU_AI_COLUMNS =

Default column mapping for Kouchou-AI format Maps Ruby gem's expected keys to Kouchou-AI's CSV column names

{
  id: "comment-id",
  body: "comment-body",
  source_url: "source-url"
}.freeze

Class Method Summary collapse

Class Method Details

.load(path, property_names: [], column_mapping: {}, encoding: "bom|utf-8") ⇒ Array<Comment>

Load comments from a CSV file



39
40
41
42
43
44
45
46
47
48
 
# File 'lib/broadlistening/csv_loader.rb', line 39

def load(path, property_names: [], column_mapping: {}, encoding: "bom|utf-8")
  mapping = KOUCHOU_AI_COLUMNS.merge(column_mapping)

  comments = []
  CSV.foreach(path, headers: true, encoding: encoding) do |row|
    comment = build_comment(row, mapping, property_names)
    comments << comment unless comment.nil?
  end
  comments
end

.parse(csv_string, property_names: [], column_mapping: {}) ⇒ Array<Comment>

Load comments from a CSV string



56
57
58
59
60
61
62
63
64
65
 
# File 'lib/broadlistening/csv_loader.rb', line 56

def parse(csv_string, property_names: [], column_mapping: {})
  mapping = KOUCHOU_AI_COLUMNS.merge(column_mapping)

  comments = []
  CSV.parse(csv_string, headers: true) do |row|
    comment = build_comment(row, mapping, property_names)
    comments << comment unless comment.nil?
  end
  comments
end