Class: CSV

Inherits:
Object
  • Object
show all
Extended by:
Forwardable
Includes:
Enumerable
Defined in:
lib/csv.rb,
lib/csv/row.rb,
lib/csv/table.rb,
lib/csv/parser.rb,
lib/csv/writer.rb,
lib/csv/match_p.rb,
lib/csv/version.rb,
lib/csv/delete_suffix.rb,
lib/csv/fields_converter.rb

Overview

This class provides a complete interface to CSV files and data. It offers tools to enable you to read and write to and from Strings or IO objects, as needed.

The most generic interface of the library is:

csv = CSV.new(io, **options)

# Reading: IO object should be open for read
csv.read # => array of rows
# or
csv.each do |row|
  # ...
end
# or
row = csv.shift

# Writing: IO object should be open for write
csv << row

There are several specialized class methods for one-statement reading or writing, described in the Specialized Methods section.

If a String is passed into ::new, it is internally wrapped into a StringIO object.

options can be used for specifying the particular CSV flavor (column separators, row separators, value quoting and so on), and for data conversion, see Data Conversion section for the description of the latter.

Specialized Methods

Reading

# From a file: all at once
arr_of_rows = CSV.read("path/to/file.csv", **options)
# iterator-style:
CSV.foreach("path/to/file.csv", **options) do |row|
  # ...
end

# From a string
arr_of_rows = CSV.parse("CSV,data,String", **options)
# or
CSV.parse("CSV,data,String", **options) do |row|
  # ...
end

Writing

# To a file
CSV.open("path/to/file.csv", "wb") do |csv|
  csv << ["row", "of", "CSV", "data"]
  csv << ["another", "row"]
  # ...
end

# To a String
csv_string = CSV.generate do |csv|
  csv << ["row", "of", "CSV", "data"]
  csv << ["another", "row"]
  # ...
end

Shortcuts

# Core extensions for converting one line
csv_string = ["CSV", "data"].to_csv   # to CSV
csv_array  = "CSV,String".parse_csv   # from CSV

# CSV() method
CSV             { |csv_out| csv_out << %w{my data here} }  # to $stdout
CSV(csv = "")   { |csv_str| csv_str << %w{my data here} }  # to a String
CSV($stderr)    { |csv_err| csv_err << %w{my data here} }  # to $stderr
CSV($stdin)     { |csv_in|  csv_in.each { |row| p row } }  # from $stdin

Options

The default values for options are:

DEFAULT_OPTIONS = {
  # For both parsing and generating.
  col_sep:            ",",
  row_sep:            :auto,
  quote_char:         '"',
  # For parsing.
  field_size_limit:   nil,
  converters:         nil,
  unconverted_fields: nil,
  headers:            false,
  return_headers:     false,
  header_converters:  nil,
  skip_blanks:        false,
  skip_lines:         nil,
  liberal_parsing:    false,
  nil_value:          nil,
  empty_value:        "",
  # For generating.
  write_headers:      nil,
  quote_empty:        true,
  force_quotes:       false,
  write_converters:   nil,
  write_nil_value:    nil,
  write_empty_value:  "",
  strip:              false,
}

Options for Parsing

:include: ../doc/col_sep.rdoc

:include: ../doc/row_sep.rdoc

:include: ../doc/quote_char.rdoc

:include: ../doc/field_size_limit.rdoc

:include: ../doc/converters.rdoc

:include: ../doc/unconverted_fields.rdoc

:include: ../doc/headers.rdoc

:include: ../doc/return_headers.rdoc

:include: ../doc/header_converters.rdoc

:include: ../doc/skip_blanks.rdoc

:include: ../doc/skip_lines.rdoc

:include: ../doc/liberal_parsing.rdoc

:include: ../doc/nil_value.rdoc

:include: ../doc/empty_value.rdoc

Options for Generating

:include: ../doc/col_sep.rdoc

:include: ../doc/row_sep.rdoc

:include: ../doc/quote_char.rdoc

:include: ../doc/write_headers.rdoc

:include: ../doc/force_quotes.rdoc

:include: ../doc/quote_empty.rdoc

:include: ../doc/write_converters.rdoc

:include: ../doc/write_nil_value.rdoc

:include: ../doc/write_empty_value.rdoc

:include: ../doc/strip.rdoc

CSV with headers

CSV allows to specify column names of CSV file, whether they are in data, or provided separately. If headers are specified, reading methods return an instance of CSV::Table, consisting of CSV::Row.

# Headers are part of data
data = CSV.parse("Name,Department,Salary\nBob,Engineering,1000\nJane,Sales,2000\nJohn,Management,5000\n", headers: true)

data.class      #=> CSV::Table
data.first      #=> #<CSV::Row "Name":"Bob" "Department":"Engineering" "Salary":"1000">
data.first.to_h #=> {"Name"=>"Bob", "Department"=>"Engineering", "Salary"=>"1000"}

# Headers provided by developer
data = CSV.parse('Bob,Engineering,1000', headers: i[name department salary])
data.first      #=> #<CSV::Row name:"Bob" department:"Engineering" salary:"1000">

CSV Converters

By default, each field parsed by CSV is formed into a String. You can use a converter to convert certain fields into other Ruby objects.

When you specify a converter for parsing, each parsed field is passed to the converter; its return value becomes the new value for the field. A converter might, for example, convert an integer embedded in a String into a true Integer. (In fact, that’s what built-in field converter :integer does.)

There are additional built-in converters, and custom converters are also supported.

All converters try to transcode fields to UTF-8 before converting. The conversion will fail if the data cannot be transcoded, leaving the field unchanged.

Field Converters

There are three ways to use field converters; these examples use built-in field converter :integer, which converts each parsed integer string to a true Integer.

Option converters with a singleton parsing method:

ary = CSV.parse_line('0,1,2', converters: :integer)
ary # => [0, 1, 2]

Option converters with a new CSV instance:

csv = CSV.new('0,1,2', converters: :integer)
# Field converters in effect:
csv.converters # => [:integer]
csv.shift # => [0, 1, 2]

Method #convert adds a field converter to a CSV instance:

csv = CSV.new('0,1,2')
# Add a converter.
csv.convert(:integer)
csv.converters # => [:integer]
csv.shift # => [0, 1, 2]

The built-in field converters are in Hash CSV::Converters. The Symbol keys there are the names of the converters:

CSV::Converters.keys # => [:integer, :float, :numeric, :date, :date_time, :all]

Converter :integer converts each field that Integer() accepts:

data = '0,1,2,x'
# Without the converter
csv = CSV.parse_line(data)
csv # => ["0", "1", "2", "x"]
# With the converter
csv = CSV.parse_line(data, converters: :integer)
csv # => [0, 1, 2, "x"]

Converter :float converts each field that Float() accepts:

data = '1.0,3.14159,x'
# Without the converter
csv = CSV.parse_line(data)
csv # => ["1.0", "3.14159", "x"]
# With the converter
csv = CSV.parse_line(data, converters: :float)
csv # => [1.0, 3.14159, "x"]

Converter :numeric converts with both :integer and :float..

Converter :date converts each field that Date::parse() accepts:

data = '2001-02-03,x'
# Without the converter
csv = CSV.parse_line(data)
csv # => ["2001-02-03", "x"]
# With the converter
csv = CSV.parse_line(data, converters: :date)
csv # => [#<Date: 2001-02-03 ((2451944j,0s,0n),+0s,2299161j)>, "x"]

Converter :date_time converts each field that +DateTime::parse() accepts:

data = '2020-05-07T14:59:00-05:00,x'
# Without the converter
csv = CSV.parse_line(data)
csv # => ["2020-05-07T14:59:00-05:00", "x"]
# With the converter
csv = CSV.parse_line(data, converters: :date_time)
csv # => [#<DateTime: 2020-05-07T14:59:00-05:00 ((2458977j,71940s,0n),-18000s,2299161j)>, "x"]

Converter :numeric converts with both :date_time and :numeric..

As seen above, method #convert adds converters to a CSV instance, and method #converters returns an Array of the converters in effect:

csv = CSV.new('0,1,2')
csv.converters # => []
csv.convert(:integer)
csv.converters # => [:integer]
csv.convert(:date)
csv.converters # => [:integer, :date]

You can add a custom field converter to Hash CSV::Converters:

strip_converter = proc {|field| field.strip}
CSV::Converters[:strip] = strip_converter
CSV::Converters.keys # => [:integer, :float, :numeric, :date, :date_time, :all, :strip]

Then use it to convert fields:

str = ' foo , 0 '
ary = CSV.parse_line(str, converters: :strip)
ary # => ["foo", "0"]

See Custom Converters.

Header Converters

Header converters operate only on headers (and not on other rows).

There are three ways to use header converters; these examples use built-in header converter :dowhcase, which downcases each parsed header.

Option header_converters with a singleton parsing method:

str = "Name,Count\nFoo,0\n,Bar,1\nBaz,2"
tbl = CSV.parse(str, headers: true, header_converters: :downcase)
tbl.class # => CSV::Table
tbl.headers # => ["name", "count"]

Option header_converters with a new CSV instance:

csv = CSV.new(str, header_converters: :downcase)
# Header converters in effect:
csv.header_converters # => [:downcase]
tbl = CSV.parse(str, headers: true)
tbl.headers # => ["Name", "Count"]

Method #header_convert adds a header converter to a CSV instance:

csv = CSV.new(str)
# Add a header converter.
csv.header_convert(:downcase)
csv.header_converters # => [:downcase]
tbl = CSV.parse(str, headers: true)
tbl.headers # => ["Name", "Count"]

The built-in header converters are in Hash CSV::Converters. The Symbol keys there are the names of the converters:

CSV::HeaderConverters.keys # => [:downcase, :symbol]

Converter :downcase converts each header by downcasing it:

str = "Name,Count\nFoo,0\n,Bar,1\nBaz,2"
tbl = CSV.parse(str, headers: true, header_converters: :downcase)
tbl.class # => CSV::Table
tbl.headers # => ["name", "count"]

Converter :symbol by making it into a Symbol:

str = "Name,Count\nFoo,0\n,Bar,1\nBaz,2"
tbl = CSV.parse(str, headers: true, header_converters: :symbol)
tbl.headers # => [:name, :count]

Details:

  • Strips leading and trailing whitespace.

  • Downcases the header.

  • Replaces embedded spaces with underscores.

  • Removes non-word characters.

  • Makes the string into a Symbol.

You can add a custom header converter to Hash CSV::HeaderConverters:

strip_converter = proc {|field| field.strip}
CSV::HeaderConverters[:strip] = strip_converter
CSV::HeaderConverters.keys # => [:downcase, :symbol, :strip]

Then use it to convert headers:

str = " Name , Value \nfoo,0\nbar,1\nbaz,2"
tbl = CSV.parse(str, headers: true, header_converters: :strip)
tbl.headers # => ["Name", "Value"]

See Custom Converters.

Custom Converters

You can define custom converters.

The converter is a Proc that is called with two arguments, String field and CSV::FieldInfo field_info; it returns a String that will become the field value:

converter = proc {|field, field_info| <some_string> }

To illustrate:

converter = proc {|field, field_info| p [field, field_info]; field}
ary = CSV.parse_line('foo,0', converters: converter)

Produces:

["foo", #<struct CSV::FieldInfo index=0, line=1, header=nil>]
["0", #<struct CSV::FieldInfo index=1, line=1, header=nil>]

In each of the output lines:

  • The first Array element is the passed String field.

  • The second is a FieldInfo structure containing information about the field:

    • The 0-based column index.

    • The 1-based line number.

    • The header for the column, if available.

If the converter does not need field_info, it can be omitted:

converter = proc {|field| ... }

CSV and Character Encodings (M17n or Multilingualization)

This new CSV parser is m17n savvy. The parser works in the Encoding of the IO or String object being read from or written to. Your data is never transcoded (unless you ask Ruby to transcode it for you) and will literally be parsed in the Encoding it is in. Thus CSV will return Arrays or Rows of Strings in the Encoding of your data. This is accomplished by transcoding the parser itself into your Encoding.

Some transcoding must take place, of course, to accomplish this multiencoding support. For example, :col_sep, :row_sep, and :quote_char must be transcoded to match your data. Hopefully this makes the entire process feel transparent, since CSV’s defaults should just magically work for your data. However, you can set these values manually in the target Encoding to avoid the translation.

It’s also important to note that while all of CSV’s core parser is now Encoding agnostic, some features are not. For example, the built-in converters will try to transcode data to UTF-8 before making conversions. Again, you can provide custom converters that are aware of your Encodings to avoid this translation. It’s just too hard for me to support native conversions in all of Ruby’s Encodings.

Anyway, the practical side of this is simple: make sure IO and String objects passed into CSV have the proper Encoding set and everything should just work. CSV methods that allow you to open IO objects (CSV::foreach(), CSV::open(), CSV::read(), and CSV::readlines()) do allow you to specify the Encoding.

One minor exception comes when generating CSV into a String with an Encoding that is not ASCII compatible. There’s no existing data for CSV to use to prepare itself and thus you will probably need to manually specify the desired Encoding for most of those cases. It will try to guess using the fields in a row of output though, when using CSV::generate_line() or Array#to_csv().

I try to point out any other Encoding issues in the documentation of methods as they come up.

This has been tested to the best of my ability with all non-“dummy” Encodings Ruby ships with. However, it is brave new code and may have some bugs. Please feel free to report any issues you find with it.

Defined Under Namespace

Modules: DeleteSuffix, MatchP Classes: FieldInfo, FieldsConverter, MalformedCSVError, Parser, Row, Table, Writer

Constant Summary collapse

DateMatcher =

A Regexp used to find and convert some common Date formats.

/ \A(?: (\w+,?\s+)?\w+\s+\d{1,2},?\s+\d{2,4} |
\d{4}-\d{2}-\d{2} )\z /x
DateTimeMatcher =

A Regexp used to find and convert some common DateTime formats.

/ \A(?: (\w+,?\s+)?\w+\s+\d{1,2}\s+\d{1,2}:\d{1,2}:\d{1,2},?\s+\d{2,4} |
    \d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2} |
    # ISO-8601
    \d{4}-\d{2}-\d{2}
      (?:T\d{2}:\d{2}(?::\d{2}(?:\.\d+)?(?:[+-]\d{2}(?::\d{2})|Z)?)?)?
)\z /x
ConverterEncoding =

The encoding used by all converters.

Encoding.find("UTF-8")
Converters =

This Hash holds the built-in converters of CSV that can be accessed by name. You can select Converters with CSV.convert() or through the options Hash passed to CSV::new().

:integer

Converts any field Integer() accepts.

:float

Converts any field Float() accepts.

:numeric

A combination of :integer and :float.

:date

Converts any field Date::parse() accepts.

:date_time

Converts any field DateTime::parse() accepts.

:all

All built-in converters. A combination of :date_time and :numeric.

All built-in converters transcode field data to UTF-8 before attempting a conversion. If your data cannot be transcoded to UTF-8 the conversion will fail and the field will remain unchanged.

This Hash is intentionally left unfrozen and users should feel free to add values to it that can be accessed by all CSV objects.

To add a combo field, the value should be an Array of names. Combo fields can be nested with other combo fields.

{
  integer:   lambda { |f|
    Integer(f.encode(ConverterEncoding)) rescue f
  },
  float:     lambda { |f|
    Float(f.encode(ConverterEncoding)) rescue f
  },
  numeric:   [:integer, :float],
  date:      lambda { |f|
    begin
      e = f.encode(ConverterEncoding)
      e.match?(DateMatcher) ? Date.parse(e) : f
    rescue  # encoding conversion or date parse errors
      f
    end
  },
  date_time: lambda { |f|
    begin
      e = f.encode(ConverterEncoding)
      e.match?(DateTimeMatcher) ? DateTime.parse(e) : f
    rescue  # encoding conversion or date parse errors
      f
    end
  },
  all:       [:date_time, :numeric],
}
HeaderConverters =

This Hash holds the built-in header converters of CSV that can be accessed by name. You can select HeaderConverters with CSV.header_convert() or through the options Hash passed to CSV::new().

:downcase

Calls downcase() on the header String.

:symbol

Leading/trailing spaces are dropped, string is downcased, remaining spaces are replaced with underscores, non-word characters are dropped, and finally to_sym() is called.

All built-in header converters transcode header data to UTF-8 before attempting a conversion. If your data cannot be transcoded to UTF-8 the conversion will fail and the header will remain unchanged.

This Hash is intentionally left unfrozen and users should feel free to add values to it that can be accessed by all CSV objects.

To add a combo field, the value should be an Array of names. Combo fields can be nested with other combo fields.

{
  downcase: lambda { |h| h.encode(ConverterEncoding).downcase },
  symbol:   lambda { |h|
    h.encode(ConverterEncoding).downcase.gsub(/[^\s\w]+/, "").strip.
                                         gsub(/\s+/, "_").to_sym
  }
}
DEFAULT_OPTIONS =

Default values for method options.

{
  # For both parsing and generating.
  col_sep:            ",",
  row_sep:            :auto,
  quote_char:         '"',
  # For parsing.
  field_size_limit:   nil,
  converters:         nil,
  unconverted_fields: nil,
  headers:            false,
  return_headers:     false,
  header_converters:  nil,
  skip_blanks:        false,
  skip_lines:         nil,
  liberal_parsing:    false,
  nil_value:          nil,
  empty_value:        "",
  # For generating.
  write_headers:      nil,
  quote_empty:        true,
  force_quotes:       false,
  write_converters:   nil,
  write_nil_value:    nil,
  write_empty_value:  "",
  strip:              false,
}.freeze
VERSION =

The version of the installed library.

"3.1.5"

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(data, col_sep: ",", row_sep: :auto, quote_char: '"', field_size_limit: nil, converters: nil, unconverted_fields: nil, headers: false, return_headers: false, write_headers: nil, header_converters: nil, skip_blanks: false, force_quotes: false, skip_lines: nil, liberal_parsing: false, internal_encoding: nil, external_encoding: nil, encoding: nil, nil_value: nil, empty_value: "", quote_empty: true, write_converters: nil, write_nil_value: nil, write_empty_value: "", strip: false) ⇒ CSV

:call-seq:

CSV.new(string)
CSV.new(io)
CSV.new(string, **options)
CSV.new(io, **options)

Returns the new CSV object created using string or io and the specified options.

Argument string should be a String object; it will be put into a new StringIO object positioned at the beginning.

Argument io should be an IO object; it will be positioned at the beginning.

To position at the end, for appending, use method CSV.generate. For any other positioning, pass a preset StringIO object instead.

In addition to the CSV instance methods, several IO methods are delegated. See CSV::open for a complete list.

For options, see:

For performance reasons, the options cannot be overridden in a CSV object, so the options specified here will endure.

Create a CSV object from a String object:

csv = CSV.new('foo,0')
csv # => #<CSV io_type:StringIO encoding:UTF-8 lineno:0 col_sep:"," row_sep:"\n" quote_char:"\"">

Create a CSV object from a File object:

File.write('t.csv', 'foo,0')
csv = CSV.new(File.open('t.csv'))
csv # => #<CSV io_type:File io_path:"t.csv" encoding:UTF-8 lineno:0 col_sep:"," row_sep:"\n" quote_char:"\"">

Raises an exception if the argument is nil:

# Raises ArgumentError (Cannot parse nil as CSV):
CSV.new(nil)

Raises:

  • (ArgumentError) 


1102
1103
1104
1105
1106
1107
1108
1109
1110
1111
1112
1113
1114
1115
1116
1117
1118
1119
1120
1121
1122
1123
1124
1125
1126
1127
1128
1129
1130
1131
1132
1133
1134
1135
1136
1137
1138
1139
1140
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
1174
1175
1176
1177
1178
1179
1180
1181
1182
1183
 
# File 'lib/csv.rb', line 1102

def initialize(data,
               col_sep: ",",
               row_sep: :auto,
               quote_char: '"',
               field_size_limit: nil,
               converters: nil,
               unconverted_fields: nil,
               headers: false,
               return_headers: false,
               write_headers: nil,
               header_converters: nil,
               skip_blanks: false,
               force_quotes: false,
               skip_lines: nil,
               liberal_parsing: false,
               internal_encoding: nil,
               external_encoding: nil,
               encoding: nil,
               nil_value: nil,
               empty_value: "",
               quote_empty: true,
               write_converters: nil,
               write_nil_value: nil,
               write_empty_value: "",
               strip: false)
  raise ArgumentError.new("Cannot parse nil as CSV") if data.nil?

  if data.is_a?(String)
    @io = StringIO.new(data)
    @io.set_encoding(encoding || data.encoding)
  else
    @io = data
  end
  @encoding = determine_encoding(encoding, internal_encoding)

  @base_fields_converter_options = {
    nil_value: nil_value,
    empty_value: empty_value,
  }
  @write_fields_converter_options = {
    nil_value: write_nil_value,
    empty_value: write_empty_value,
  }
  @initial_converters = converters
  @initial_header_converters = header_converters
  @initial_write_converters = write_converters

  @parser_options = {
    column_separator: col_sep,
    row_separator: row_sep,
    quote_character: quote_char,
    field_size_limit: field_size_limit,
    unconverted_fields: unconverted_fields,
    headers: headers,
    return_headers: return_headers,
    skip_blanks: skip_blanks,
    skip_lines: skip_lines,
    liberal_parsing: liberal_parsing,
    encoding: @encoding,
    nil_value: nil_value,
    empty_value: empty_value,
    strip: strip,
  }
  @parser = nil
  @parser_enumerator = nil
  @eof_error = nil

  @writer_options = {
    encoding: @encoding,
    force_encoding: (not encoding.nil?),
    force_quotes: force_quotes,
    headers: headers,
    write_headers: write_headers,
    column_separator: col_sep,
    row_separator: row_sep,
    quote_character: quote_char,
    quote_empty: quote_empty,
  }

  @writer = nil
  writer if @writer_options[:write_headers]
end

Instance Attribute Details

#encoding ⇒ Object (readonly)

The Encoding CSV is parsing or writing in. This will be the Encoding you receive parsed data in and/or the Encoding data will be written in.



1311
1312
1313
 
# File 'lib/csv.rb', line 1311

def encoding
  @encoding
end

Class Method Details

.filter(input = nil, output = nil, **options) ⇒ Object

:call-seq:

filter( **options ) { |row| ... }
filter( input, **options ) { |row| ... }
filter( input, output, **options ) { |row| ... }

This method is a convenience for building Unix-like filters for CSV data. Each row is yielded to the provided block which can alter it as needed. After the block returns, the row is appended to output altered or not.

The input and output arguments can be anything CSV::new() accepts (generally String or IO objects). If not given, they default to ARGF and $stdout.

The options parameter is also filtered down to CSV::new() after some clever key parsing. Any key beginning with :in_ or :input_ will have that leading identifier stripped and will only be used in the options Hash for the input object. Keys starting with :out_ or :output_ affect only output. All other keys are assigned to both objects.

See Options for Parsing and Options for Generating.

The :output_row_sep option defaults to $INPUT_RECORD_SEPARATOR ($/).



729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
 
# File 'lib/csv.rb', line 729

def filter(input=nil, output=nil, **options)
  # parse options for input, output, or both
  in_options, out_options = Hash.new, {row_sep: $INPUT_RECORD_SEPARATOR}
  options.each do |key, value|
    case key.to_s
    when /\Ain(?:put)?_(.+)\Z/
      in_options[$1.to_sym] = value
    when /\Aout(?:put)?_(.+)\Z/
      out_options[$1.to_sym] = value
    else
      in_options[key]  = value
      out_options[key] = value
    end
  end
  # build input and output wrappers
  input  = new(input  || ARGF,    **in_options)
  output = new(output || $stdout, **out_options)

  # read, yield, write
  input.each do |row|
    yield row
    output << row
  end
end

.foreach(path, mode = "r", **options, &block) ⇒ Object

This method is intended as the primary interface for reading CSV files. You pass a path and any options you wish to set for the read. Each row of file will be passed to the provided block in turn.

See Options for Parsing.

The options parameter can be anything CSV::new() understands. This method also understands an additional :encoding parameter that you can use to specify the Encoding of the data in the file to be read. You must provide this unless your data is in Encoding::default_external(). CSV will use this to determine how to parse the data. You may provide a second Encoding to have the data transcoded as it is read. For example, encoding: "UTF-32BE:UTF-8" would read UTF-32BE data from the file but transcode it to UTF-8 before CSV parses it.



770
771
772
773
774
775
 
# File 'lib/csv.rb', line 770

def foreach(path, mode="r", **options, &block)
  return to_enum(__method__, path, mode, **options) unless block_given?
  open(path, mode, **options) do |csv|
    csv.each(&block)
  end
end

.generate(str = nil, **options) {|csv| ... } ⇒ Object

:call-seq:

generate( str, **options ) { |csv| ... }
generate( **options ) { |csv| ... }

This method wraps a String you provide, or an empty default String, in a CSV object which is passed to the provided block. You can use the block to append CSV rows to the String and when the block exits, the final String will be returned.

Note that a passed String is modified by this method. Call dup() before passing if you need a new String.

See Options for Generating.

This method has one additional option: :encoding, which sets the base Encoding for the output if no no str is specified. CSV needs this hint if you plan to output non-ASCII compatible data.

Yields:

  • (csv) 


796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
 
# File 'lib/csv.rb', line 796

def generate(str=nil, **options)
  encoding = options[:encoding]
  # add a default empty String, if none was given
  if str
    str = StringIO.new(str)
    str.seek(0, IO::SEEK_END)
    str.set_encoding(encoding) if encoding
  else
    str = +""
    str.force_encoding(encoding) if encoding
  end
  csv = new(str, **options) # wrap
  yield csv         # yield for appending
  csv.string        # return final String
end

.generate_line(row, **options) ⇒ Object

:call-seq:

CSV.generate_line(ary)
CSV.generate_line(ary, **options)

Returns the String created by generating CSV from ary using the specified options.

Argument ary must be an Array.

Special options:

  • Option :row_sep defaults to $INPUT_RECORD_SEPARATOR ($/).:

    $INPUT_RECORD_SEPARATOR # => "\n"

  • This method accepts an additional option, :encoding, which sets the base Encoding for the output. This method will try to guess your Encoding from the first non-nil field in row, if possible, but you may need to use this parameter as a backup plan.

For other options, see Options for Generating.

Returns the String generated from an Array:

CSV.generate_line(['foo', '0']) # => "foo,0\n"

Raises an exception if ary is not an Array:

# Raises NoMethodError (undefined method `find' for :foo:Symbol)
CSV.generate_line(:foo)



844
845
846
847
848
849
850
851
852
853
 
# File 'lib/csv.rb', line 844

def generate_line(row, **options)
  options = {row_sep: $INPUT_RECORD_SEPARATOR}.merge(options)
  str = +""
  if options[:encoding]
    str.force_encoding(options[:encoding])
  elsif field = row.find {|f| f.is_a?(String)}
    str.force_encoding(field.encoding)
  end
  (new(str, **options) << row).string
end

.instance(data = $stdout, **options) ⇒ Object

This method will return a CSV instance, just like CSV::new(), but the instance will be cached and returned for all future calls to this method for the same data object (tested by Object#object_id()) with the same options.

See Options for Parsing and Options for Generating.

If a block is given, the instance is passed to the block and the return value becomes the return value of the block.



686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
 
# File 'lib/csv.rb', line 686

def instance(data = $stdout, **options)
  # create a _signature_ for this method call, data object and options
  sig = [data.object_id] +
        options.values_at(*DEFAULT_OPTIONS.keys.sort_by { |sym| sym.to_s })

  # fetch or create the instance for this signature
  @@instances ||= Hash.new
  instance = (@@instances[sig] ||= new(data, **options))

  if block_given?
    yield instance  # run block, if given, returning result
  else
    instance        # or return the instance
  end
end

.open(filename, mode = "r", **options) ⇒ Object

:call-seq:

open( filename, mode = "rb", **options ) { |faster_csv| ... }
open( filename, **options ) { |faster_csv| ... }
open( filename, mode = "rb", **options )
open( filename, **options )

This method opens an IO object, and wraps that with CSV. This is intended as the primary interface for writing a CSV file.

You must pass a filename and may optionally add a mode for Ruby’s open().

See Options for Generating.

This method works like Ruby’s open() call, in that it will pass a CSV object to a provided block and close it when the block terminates, or it will return the CSV object when no block is provided. (Note: This is different from the Ruby 1.8 CSV library which passed rows to the block. Use CSV::foreach() for that behavior.)

You must provide a mode with an embedded Encoding designator unless your data is in Encoding::default_external(). CSV will check the Encoding of the underlying IO object (set by the mode you pass) to determine how to parse the data. You may provide a second Encoding to have the data transcoded as it is read just as you can with a normal call to IO::open(). For example, "rb:UTF-32BE:UTF-8" would read UTF-32BE data from the file but transcode it to UTF-8 before CSV parses it.

An opened CSV object will delegate to many IO methods for convenience. You may call:

  • binmode()

  • binmode?()

  • close()

  • close_read()

  • close_write()

  • closed?()

  • eof()

  • eof?()

  • external_encoding()

  • fcntl()

  • fileno()

  • flock()

  • flush()

  • fsync()

  • internal_encoding()

  • ioctl()

  • isatty()

  • path()

  • pid()

  • pos()

  • pos=()

  • reopen()

  • seek()

  • stat()

  • sync()

  • sync=()

  • tell()

  • to_i()

  • to_io()

  • truncate()

  • tty?()



919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
 
# File 'lib/csv.rb', line 919

def open(filename, mode="r", **options)
  # wrap a File opened with the remaining +args+ with no newline
  # decorator
  file_opts = {universal_newline: false}.merge(options)

  begin
    f = File.open(filename, mode, **file_opts)
  rescue ArgumentError => e
    raise unless /needs binmode/.match?(e.message) and mode == "r"
    mode = "rb"
    file_opts = {encoding: Encoding.default_external}.merge(file_opts)
    retry
  end
  begin
    csv = new(f, **options)
  rescue Exception
    f.close
    raise
  end

  # handle blocks like Ruby's open(), not like the CSV library
  if block_given?
    begin
      yield csv
    ensure
      csv.close
    end
  else
    csv
  end
end

.parse(str, **options, &block) ⇒ Object

:call-seq:

parse( str, **options ) { |row| ... }
parse( str, **options )

This method can be used to easily parse CSV out of a String. You may either provide a block which will be called with each row of the String in turn, or just use the returned Array of Arrays (when no block is given).

You pass your str to read from, and an optional options. See Options for Parsing.



963
964
965
966
967
968
969
970
971
972
973
974
 
# File 'lib/csv.rb', line 963

def parse(str, **options, &block)
  csv = new(str, **options)

  return csv.each(&block) if block_given?

  # slurp contents, if no block is given
  begin
    csv.read
  ensure
    csv.close
  end
end

.parse_line(line, **options) ⇒ Object

:call-seq:

CSV.parse_line(string)
CSV.parse_line(io)
CSV.parse_line(string, **options)
CSV.parse_line(io, **options)

Returns the new Array created by parsing the first line of string or io using the specified options.

Argument string should be a String object; it will be put into a new StringIO object positioned at the beginning.

Argument io should be an IO object; it will be positioned at the beginning.

For options, see Options for Parsing.

Returns data from the first line from a String object:

CSV.parse_line('foo,0') # => ["foo", "0"]

Returns data from the first line from a File object:

File.write('t.csv', 'foo,0')
CSV.parse_line(File.open('t.csv')) # => ["foo", "0"]

Ignores lines after the first:

CSV.parse_line("foo,0\nbar,1\nbaz,2") # => ["foo", "0"]

Returns nil if the argument is an empty String:

CSV.parse_line('') # => nil

Raises an exception if the argument is nil:

# Raises ArgumentError (Cannot parse nil as CSV):
CSV.parse_line(nil)



1012
1013
1014
 
# File 'lib/csv.rb', line 1012

def parse_line(line, **options)
  new(line, **options).each.first
end

.read(path, **options) ⇒ Object

Use to slurp a CSV file into an Array of Arrays. Pass the path to the file and options. See Options for Parsing.

This method also understands an additional :encoding parameter that you can use to specify the Encoding of the data in the file to be read. You must provide this unless your data is in Encoding::default_external(). CSV will use this to determine how to parse the data. You may provide a second Encoding to have the data transcoded as it is read. For example, encoding: "UTF-32BE:UTF-8" would read UTF-32BE data from the file but transcode it to UTF-8 before CSV parses it.



1030
1031
1032
 
# File 'lib/csv.rb', line 1030

def read(path, **options)
  open(path, **options) { |csv| csv.read }
end

.readlines(path, **options) ⇒ Object

Alias for CSV::read().



1035
1036
1037
 
# File 'lib/csv.rb', line 1035

def readlines(path, **options)
  read(path, **options)
end

.table(path, **options) ⇒ Object

A shortcut for:

CSV.read( path, { headers:           true,
                  converters:        :numeric,
                  header_converters: :symbol }.merge(options) )

See Options for Parsing.



1047
1048
1049
1050
1051
1052
1053
1054
1055
 
# File 'lib/csv.rb', line 1047

def table(path, **options)
  default_options = {
    headers:           true,
    converters:        :numeric,
    header_converters: :symbol,
  }
  options = default_options.merge(options)
  read(path, **options)
end

Instance Method Details

#<<(row) ⇒ Object Also known as: add_row, puts

The primary write method for wrapped Strings and IOs, row (an Array or CSV::Row) is converted to CSV and appended to the data source. When a CSV::Row is passed, only the row’s fields() are appended to the output.

The data source must be open for writing.



1410
1411
1412
1413
 
# File 'lib/csv.rb', line 1410

def <<(row)
  writer << row
  self
end

#binmode? ⇒ Boolean

Returns:

  • (Boolean) 


1342
1343
1344
1345
1346
1347
1348
 
# File 'lib/csv.rb', line 1342

def binmode?
  if @io.respond_to?(:binmode?)
    @io.binmode?
  else
    false
  end
end

#col_sep ⇒ Object

The encoded :col_sep used in parsing and writing. See CSV::new for details.



1189
1190
1191
 
# File 'lib/csv.rb', line 1189

def col_sep
  parser.column_separator
end

#convert(name = nil, &converter) ⇒ Object

:call-seq:

convert( name )
convert { |field| ... }
convert { |field, field_info| ... }

You can use this method to install a CSV::Converters built-in, or provide a block that handles a custom conversion.

If you provide a block that takes one argument, it will be passed the field and is expected to return the converted value or the field itself. If your block takes two arguments, it will also be passed a CSV::FieldInfo Struct, containing details about the field. Again, the block should return a converted field or the field itself.



1432
1433
1434
 
# File 'lib/csv.rb', line 1432

def convert(name = nil, &converter)
  parser_fields_converter.add_converter(name, &converter)
end

#converters ⇒ Object

Returns the current list of converters in effect. See CSV::new for details. Built-in converters will be returned by name, while others will be returned as is.



1230
1231
1232
1233
1234
1235
 
# File 'lib/csv.rb', line 1230

def converters
  parser_fields_converter.map do |converter|
    name = Converters.rassoc(converter)
    name ? name.first : converter
  end
end

#each(&block) ⇒ Object

Yields each row of the data source in turn.

Support for Enumerable.

The data source must be open for reading.



1460
1461
1462
 
# File 'lib/csv.rb', line 1460

def each(&block)
  parser_enumerator.each(&block)
end

#eof? ⇒ Boolean Also known as: eof

Returns:

  • (Boolean) 


1378
1379
1380
1381
1382
1383
1384
1385
1386
1387
1388
1389
 
# File 'lib/csv.rb', line 1378

def eof?
  return false if @eof_error
  begin
    parser_enumerator.peek
    false
  rescue MalformedCSVError => error
    @eof_error = error
    false
  rescue StopIteration
    true
  end
end

#field_size_limit ⇒ Object

The limit for field size, if any. See CSV::new for details.



1213
1214
1215
 
# File 'lib/csv.rb', line 1213

def field_size_limit
  parser.field_size_limit
end

#flock(*args) ⇒ Object

Raises:

  • (NotImplementedError) 


1350
1351
1352
1353
 
# File 'lib/csv.rb', line 1350

def flock(*args)
  raise NotImplementedError unless @io.respond_to?(:flock)
  @io.flock(*args)
end

#force_quotes? ⇒ Boolean

Returns true if all output fields are quoted. See CSV::new for details.

Returns:

  • (Boolean) 


1298
1299
1300
 
# File 'lib/csv.rb', line 1298

def force_quotes?
  @writer_options[:force_quotes]
end

#header_convert(name = nil, &converter) ⇒ Object

:call-seq:

header_convert( name )
header_convert { |field| ... }
header_convert { |field, field_info| ... }

Identical to CSV#convert(), but for header rows.

Note that this method must be called before header rows are read to have any effect.



1447
1448
1449
 
# File 'lib/csv.rb', line 1447

def header_convert(name = nil, &converter)
  header_fields_converter.add_converter(name, &converter)
end

#header_converters ⇒ Object

Returns the current list of converters in effect for headers. See CSV::new for details. Built-in converters will be returned by name, while others will be returned as is.



1282
1283
1284
1285
1286
1287
 
# File 'lib/csv.rb', line 1282

def header_converters
  header_fields_converter.map do |converter|
    name = HeaderConverters.rassoc(converter)
    name ? name.first : converter
  end
end

#header_row? ⇒ Boolean

Returns true if the next row read will be a header row.

Returns:

  • (Boolean) 


1480
1481
1482
 
# File 'lib/csv.rb', line 1480

def header_row?
  parser.header_row?
end

#headers ⇒ Object

Returns nil if headers will not be used, true if they will but have not yet been read, or the actual headers after they have been read. See CSV::new for details.



1250
1251
1252
1253
1254
1255
1256
1257
1258
1259
1260
 
# File 'lib/csv.rb', line 1250

def headers
  if @writer
    @writer.headers
  else
    parsed_headers = parser.headers
    return parsed_headers if parsed_headers
    raw_headers = @parser_options[:headers]
    raw_headers = nil if raw_headers == false
    raw_headers
  end
end

#inspect ⇒ Object

Returns a simplified description of the key CSV attributes in an ASCII compatible String.



1509
1510
1511
1512
1513
1514
1515
1516
1517
1518
1519
1520
1521
1522
1523
1524
1525
1526
1527
1528
1529
1530
1531
1532
1533
1534
1535
1536
1537
1538
1539
1540
1541
1542
1543
1544
1545
 
# File 'lib/csv.rb', line 1509

def inspect
  str = ["#<", self.class.to_s, " io_type:"]
  # show type of wrapped IO
  if    @io == $stdout then str << "$stdout"
  elsif @io == $stdin  then str << "$stdin"
  elsif @io == $stderr then str << "$stderr"
  else                      str << @io.class.to_s
  end
  # show IO.path(), if available
  if @io.respond_to?(:path) and (p = @io.path)
    str << " io_path:" << p.inspect
  end
  # show encoding
  str << " encoding:" << @encoding.name
  # show other attributes
  ["lineno", "col_sep", "row_sep", "quote_char"].each do |attr_name|
    if a = __send__(attr_name)
      str << " " << attr_name << ":" << a.inspect
    end
  end
  ["skip_blanks", "liberal_parsing"].each do |attr_name|
    if a = __send__("#{attr_name}?")
      str << " " << attr_name << ":" << a.inspect
    end
  end
  _headers = headers
  str << " headers:" << _headers.inspect if _headers
  str << ">"
  begin
    str.join('')
  rescue  # any encoding error
    str.map do |s|
      e = Encoding::Converter.asciicompat_encoding(s.encoding)
      e ? s.encode(e) : s.force_encoding("ASCII-8BIT")
    end.join('')
  end
end

#ioctl(*args) ⇒ Object

Raises:

  • (NotImplementedError) 


1355
1356
1357
1358
 
# File 'lib/csv.rb', line 1355

def ioctl(*args)
  raise NotImplementedError unless @io.respond_to?(:ioctl)
  @io.ioctl(*args)
end

#liberal_parsing? ⇒ Boolean

Returns true if illegal input is handled. See CSV::new for details.

Returns:

  • (Boolean) 


1303
1304
1305
 
# File 'lib/csv.rb', line 1303

def liberal_parsing?
  parser.liberal_parsing?
end

#line ⇒ Object

The last row read from this file.



1328
1329
1330
 
# File 'lib/csv.rb', line 1328

def line
  parser.line
end

#lineno ⇒ Object

The line number of the last row read from this file. Fields with nested line-end characters will not affect this count.



1317
1318
1319
1320
1321
1322
1323
 
# File 'lib/csv.rb', line 1317

def lineno
  if @writer
    @writer.lineno
  else
    parser.lineno
  end
end

#path ⇒ Object 



1360
1361
1362
 
# File 'lib/csv.rb', line 1360

def path
  @io.path if @io.respond_to?(:path)
end

#quote_char ⇒ Object

The encoded :quote_char used in parsing and writing. See CSV::new for details.



1205
1206
1207
 
# File 'lib/csv.rb', line 1205

def quote_char
  parser.quote_character
end

#read ⇒ Object Also known as: readlines

Slurps the remaining rows and returns an Array of Arrays.

The data source must be open for reading.



1469
1470
1471
1472
1473
1474
1475
1476
 
# File 'lib/csv.rb', line 1469

def read
  rows = to_a
  if parser.use_headers?
    Table.new(rows, headers: parser.headers)
  else
    rows
  end
end

#return_headers? ⇒ Boolean

Returns true if headers will be returned as a row of results. See CSV::new for details.

Returns:

  • (Boolean) 


1265
1266
1267
 
# File 'lib/csv.rb', line 1265

def return_headers?
  parser.return_headers?
end

#rewind ⇒ Object

Rewinds the underlying IO object and resets CSV’s lineno() counter.



1393
1394
1395
1396
1397
1398
1399
 
# File 'lib/csv.rb', line 1393

def rewind
  @parser = nil
  @parser_enumerator = nil
  @eof_error = nil
  @writer.rewind if @writer
  @io.rewind
end

#row_sep ⇒ Object

The encoded :row_sep used in parsing and writing. See CSV::new for details.



1197
1198
1199
 
# File 'lib/csv.rb', line 1197

def row_sep
  parser.row_separator
end

#shift ⇒ Object Also known as: gets, readline

The primary read method for wrapped Strings and IOs, a single row is pulled from the data source, parsed and returned as an Array of fields (if header rows are not used) or a CSV::Row (when header rows are used).

The data source must be open for reading.



1491
1492
1493
1494
1495
1496
1497
1498
1499
1500
1501
 
# File 'lib/csv.rb', line 1491

def shift
  if @eof_error
    eof_error, @eof_error = @eof_error, nil
    raise eof_error
  end
  begin
    parser_enumerator.next
  rescue StopIteration
    nil
  end
end

#skip_blanks? ⇒ Boolean

Returns true blank lines are skipped by the parser. See CSV::new for details.

Returns:

  • (Boolean) 


1293
1294
1295
 
# File 'lib/csv.rb', line 1293

def skip_blanks?
  parser.skip_blanks?
end

#skip_lines ⇒ Object

The regex marking a line as a comment. See CSV::new for details.



1221
1222
1223
 
# File 'lib/csv.rb', line 1221

def skip_lines
  parser.skip_lines
end

#stat(*args) ⇒ Object

Raises:

  • (NotImplementedError) 


1364
1365
1366
1367
 
# File 'lib/csv.rb', line 1364

def stat(*args)
  raise NotImplementedError unless @io.respond_to?(:stat)
  @io.stat(*args)
end

#to_i ⇒ Object

Raises:

  • (NotImplementedError) 


1369
1370
1371
1372
 
# File 'lib/csv.rb', line 1369

def to_i
  raise NotImplementedError unless @io.respond_to?(:to_i)
  @io.to_i
end

#to_io ⇒ Object 



1374
1375
1376
 
# File 'lib/csv.rb', line 1374

def to_io
  @io.respond_to?(:to_io) ? @io.to_io : @io
end

#unconverted_fields? ⇒ Boolean

Returns true if unconverted_fields() to parsed results. See CSV::new for details.

Returns:

  • (Boolean) 


1241
1242
1243
 
# File 'lib/csv.rb', line 1241

def unconverted_fields?
  parser.unconverted_fields?
end

#write_headers? ⇒ Boolean

Returns true if headers are written in output. See CSV::new for details.

Returns:

  • (Boolean) 


1273
1274
1275
 
# File 'lib/csv.rb', line 1273

def write_headers?
  @writer_options[:write_headers]
end