Class: FasterCSV

Inherits:
Object
  • Object
show all
Extended by:
Forwardable
Includes:
Enumerable
Defined in:
lib/faster_csv.rb,
lib/faster_csv.rb

Overview

This class provides a complete interface to CSV files and data. It offers tools to enable you to read and write to and from Strings or IO objects, as needed.

Reading

From a File

A Line at a Time

FasterCSV.foreach("path/to/file.csv") do |row|
  # use row here...
end

All at Once

arr_of_arrs = FasterCSV.read("path/to/file.csv")

From a String

A Line at a Time

FasterCSV.parse("CSV,data,String") do |row|
  # use row here...
end

All at Once

arr_of_arrs = FasterCSV.parse("CSV,data,String")

Writing

To a File

FasterCSV.open("path/to/file.csv", "w") do |csv|
  csv << ["row", "of", "CSV", "data"]
  csv << ["another", "row"]
  # ...
end

To a String

csv_string = FasterCSV.generate do |csv|
  csv << ["row", "of", "CSV", "data"]
  csv << ["another", "row"]
  # ...
end

Convert a Single Line

csv_string = ["CSV", "data"].to_csv   # to CSV
csv_array  = "CSV,String".parse_csv   # from CSV

Shortcut Interface

FCSV             { |csv_out| csv_out << %w{my data here} }  # to $stdout
FCSV(csv = "")   { |csv_str| csv_str << %w{my data here} }  # to a String
FCSV($stderr)    { |csv_err| csv_err << %w{my data here} }  # to $stderr
FCSV($stdin)     { |csv_in|  csv_in.each { |row| p row } }  # from $stdin

Advanced Usage

Wrap an IO Object

csv = FCSV.new(io, options)
# ... read (with gets() or each()) from and write (with <<) to csv here ...

Defined Under Namespace

Classes: FieldInfo, MalformedCSVError, Row, Table

Constant Summary collapse

VERSION =

The version of the installed library.

"1.5.5".freeze
DateMatcher =

A Regexp used to find and convert some common Date formats.

/ \A(?: (\w+,?\s+)?\w+\s+\d{1,2},?\s+\d{2,4} |
\d{4}-\d{2}-\d{2} )\z /x
DateTimeMatcher =

A Regexp used to find and convert some common DateTime formats.

/ \A(?: (\w+,?\s+)?\w+\s+\d{1,2}\s+\d{1,2}:\d{1,2}:\d{1,2},?\s+\d{2,4} |
\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2} )\z /x
Converters =

This Hash holds the built-in converters of FasterCSV that can be accessed by name. You can select Converters with FasterCSV.convert() or through the options Hash passed to FasterCSV::new().

:integer

Converts any field Integer() accepts.

:float

Converts any field Float() accepts.

:numeric

A combination of :integer and :float.

:date

Converts any field Date::parse() accepts.

:date_time

Converts any field DateTime::parse() accepts.

:all

All built-in converters. A combination of :date_time and :numeric.

This Hash is intetionally left unfrozen and users should feel free to add values to it that can be accessed by all FasterCSV objects.

To add a combo field, the value should be an Array of names. Combo fields can be nested with other combo fields.

{ :integer   => lambda { |f| Integer(f)        rescue f },
:float     => lambda { |f| Float(f)          rescue f },
:numeric   => [:integer, :float],
:date      => lambda { |f|
  f =~ DateMatcher ? (Date.parse(f) rescue f) : f
},
:date_time => lambda { |f|
  f =~ DateTimeMatcher ? (DateTime.parse(f) rescue f) : f
},
:all       => [:date_time, :numeric] }
HeaderConverters =

This Hash holds the built-in header converters of FasterCSV that can be accessed by name. You can select HeaderConverters with FasterCSV.header_convert() or through the options Hash passed to FasterCSV::new().

:downcase

Calls downcase() on the header String.

:symbol

The header String is downcased, spaces are replaced with underscores, non-word characters are dropped, and finally to_sym() is called.

This Hash is intetionally left unfrozen and users should feel free to add values to it that can be accessed by all FasterCSV objects.

To add a combo field, the value should be an Array of names. Combo fields can be nested with other combo fields.

{
  :downcase => lambda { |h| h.downcase },
  :symbol   => lambda { |h|
    h.downcase.tr(" ", "_").delete("^a-z0-9_").to_sym
  }
}
DEFAULT_OPTIONS =

The options used when no overrides are given by calling code. They are:

:col_sep

","

:row_sep

:auto

:quote_char

'"'

:converters

nil

:unconverted_fields

nil

:headers

false

:return_headers

false

:header_converters

nil

:skip_blanks

false

:force_quotes

false

{ :col_sep            => ",",
:row_sep            => :auto,
:quote_char         => '"', 
:converters         => nil,
:unconverted_fields => nil,
:headers            => false,
:return_headers     => false,
:header_converters  => nil,
:skip_blanks        => false,
:force_quotes       => false }.freeze

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(data, options = Hash.new) ⇒ FasterCSV

This constructor will wrap either a String or IO object passed in data for reading and/or writing. In addition to the FasterCSV instance methods, several IO methods are delegated. (See FasterCSV::open() for a complete list.) If you pass a String for data, you can later retrieve it (after writing to it, for example) with FasterCSV.string().

Note that a wrapped String will be positioned at the beginning (for reading). If you want it at the end (for writing), use FasterCSV::generate(). If you want any other positioning, pass a preset StringIO object instead.

You may set any reading and/or writing preferences in the options Hash.

Available options are:

:col_sep

The String placed between each field.

:row_sep

The String appended to the end of each row. This can be set to the special :auto setting, which requests that FasterCSV automatically discover this from the data. Auto-discovery reads ahead in the data looking for the next "\r\n", "\n", or "\r" sequence. A sequence will be selected even if it occurs in a quoted field, assuming that you would have the same line endings there. If none of those sequences is found, or the stream is only available for output, the default $INPUT_RECORD_SEPARATOR ($/) is used. Obviously, discovery takes a little time. Set manually if speed is important. Also note that IO objects should be opened in binary mode on Windows if this feature will be used as the line-ending translation can cause problems with resetting the document position to where it was before the read ahead.

:quote_char

The character used to quote fields. This has to be a single character String. This is useful for application that incorrectly use ' as the quote character instead of the correct ". FasterCSV will always consider a double sequence this character to be an escaped quote.

:encoding

The encoding to use when parsing the file. Defaults to your $KCODE setting. Valid values: `n’ or `N’ for none, `e’ or `E’ for EUC, `s’ or `S’ for SJIS, and `u’ or `U’ for UTF-8 (see Regexp.new()).

:field_size_limit

This is a maximum size FasterCSV will read ahead looking for the closing quote for a field. (In truth, it reads to the first line ending beyond this size.) If a quote cannot be found within the limit FasterCSV will raise a MalformedCSVError, assuming the data is faulty. You can use this limit to prevent what are effectively DoS attacks on the parser. However, this limit can cause a legitimate parse to fail and thus is set to nil, or off, by default.

:converters

An Array of names from the Converters Hash and/or lambdas that handle custom conversion. A single converter doesn’t have to be in an Array.

:unconverted_fields

If set to true, an unconverted_fields() method will be added to all returned rows (Array or FasterCSV::Row) that will return the fields as they were before convertion. Note that :headers supplied by Array or String were not fields of the document and thus will have an empty Array attached.

:headers

If set to :first_row or true, the initial row of the CSV file will be treated as a row of headers. If set to an Array, the contents will be used as the headers. If set to a String, the String is run through a call of FasterCSV::parse_line() with the same :col_sep, :row_sep, and :quote_char as this instance to produce an Array of headers. This setting causes FasterCSV.shift() to return rows as FasterCSV::Row objects instead of Arrays and FasterCSV.read() to return FasterCSV::Table objects instead of an Array of Arrays.

:return_headers

When false, header rows are silently swallowed. If set to true, header rows are returned in a FasterCSV::Row object with identical headers and fields (save that the fields do not go through the converters).

:write_headers

When true and :headers is set, a header row will be added to the output. Note that if the table only contains header rows, :return_headers must also be set in order for a header row to be output.

:header_converters

Identical in functionality to :converters save that the conversions are only made to header rows.

:skip_blanks

When set to a true value, FasterCSV will skip over any rows with no content.

:force_quotes

When set to a true value, FasterCSV will quote all CSV fields it creates.

See FasterCSV::DEFAULT_OPTIONS for the default settings.

Options cannot be overriden in the instance methods for performance reasons, so be sure to set what you want here.



1423
1424
1425
1426
1427
1428
1429
1430
1431
1432
1433
1434
1435
1436
1437
1438
1439
1440
1441
 
# File 'lib/faster_csv.rb', line 1423

def initialize(data, options = Hash.new)
  # build the options for this read/write
  options = DEFAULT_OPTIONS.merge(options)

  # create the IO object we will read from
  @io = if data.is_a? String then StringIO.new(data) else data end

  init_separators(options)
  init_parsers(options)
  init_converters(options)
  init_headers(options)

  unless options.empty?
    raise ArgumentError, "Unknown options:  #{options.keys.join(', ')}."
  end

  # track our own lineno since IO gets confused about line-ends is CSV fields
  @lineno = 0
end

Dynamic Method Handling

This class handles dynamic methods through the method_missing method

#method_missing(*_) ⇒ Object 



22
23
24
 
# File 'lib/faster_csv.rb', line 22

def method_missing(*_)
  self.class.const_missing
end

Instance Attribute Details

#lineno ⇒ Object (readonly)

The line number of the last row read from this file. Fields with nested line-end characters will not affect this count.



1447
1448
1449
 
# File 'lib/faster_csv.rb', line 1447

def lineno
  @lineno
end

Class Method Details

.build_csv_interface ⇒ Object

This method will build a drop-in replacement for many of the standard CSV methods. It allows you to write code like:

begin
  require "faster_csv"
  FasterCSV.build_csv_interface
rescue LoadError
  require "csv"
end
# ... use CSV here ...

This is not a complete interface with completely identical behavior. However, it is intended to be close enough that you won’t notice the difference in most cases. CSV methods supported are:

  • foreach()

  • generate_line()

  • open()

  • parse()

  • parse_line()

  • readlines()

Be warned that this interface is slower than vanilla FasterCSV due to the extra layer of method calls. Depending on usage, this can slow it down to near CSV speeds.



874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
 
# File 'lib/faster_csv.rb', line 874

def self.build_csv_interface
  Object.const_set(:CSV, Class.new).class_eval do
    def self.foreach(path, rs = :auto, &block)  # :nodoc:
      FasterCSV.foreach(path, :row_sep => rs, &block)
    end

    def self.generate_line(row, fs = ",", rs = "")  # :nodoc:
      FasterCSV.generate_line(row, :col_sep => fs, :row_sep => rs)
    end

    def self.open(path, mode, fs = ",", rs = :auto, &block)  # :nodoc:
      if block and mode.include? "r"
        FasterCSV.open(path, mode, :col_sep => fs, :row_sep => rs) do |csv|
          csv.each(&block)
        end
      else
        FasterCSV.open(path, mode, :col_sep => fs, :row_sep => rs, &block)
      end
    end

    def self.parse(str_or_readable, fs = ",", rs = :auto, &block)  # :nodoc:
      FasterCSV.parse(str_or_readable, :col_sep => fs, :row_sep => rs, &block)
    end

    def self.parse_line(src, fs = ",", rs = :auto)  # :nodoc:
      FasterCSV.parse_line(src, :col_sep => fs, :row_sep => rs)
    end

    def self.readlines(path, rs = :auto)  # :nodoc:
      FasterCSV.readlines(path, :row_sep => rs)
    end
  end
end

.const_missing(*_) ⇒ Object

Raises:

  • (NotImplementedError) 


12
13
14
15
16
 
# File 'lib/faster_csv.rb', line 12

def self.const_missing(*_)
  raise NotImplementedError, "Please switch to Ruby 1.9's standard CSV "  +
                             "library.  It's FasterCSV plus support for " +
                             "Ruby 1.9's m17n encoding engine."
end

.dump(ary_of_objs, io = "", options = Hash.new) ⇒ Object

This method allows you to serialize an Array of Ruby objects to a String or File of CSV data. This is not as powerful as Marshal or YAML, but perhaps useful for spreadsheet and database interaction.

Out of the box, this method is intended to work with simple data objects or Structs. It will serialize a list of instance variables and/or Struct.members().

If you need need more complicated serialization, you can control the process by adding methods to the class to be serialized.

A class method csv_meta() is responsible for returning the first row of the document (as an Array). This row is considered to be a Hash of the form key_1,value_1,key_2,value_2,… FasterCSV::load() expects to find a class key with a value of the stringified class name and FasterCSV::dump() will create this, if you do not define this method. This method is only called on the first object of the Array.

The next method you can provide is an instance method called csv_headers(). This method is expected to return the second line of the document (again as an Array), which is to be used to give each column a header. By default, FasterCSV::load() will set an instance variable if the field header starts with an @ character or call send() passing the header as the method name and the field value as an argument. This method is only called on the first object of the Array.

Finally, you can provide an instance method called csv_dump(), which will be passed the headers. This should return an Array of fields that can be serialized for this object. This method is called once for every object in the Array.

The io parameter can be used to serialize to a File, and options can be anything FasterCSV::new() accepts.



943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
 
# File 'lib/faster_csv.rb', line 943

def self.dump(ary_of_objs, io = "", options = Hash.new)
  obj_template = ary_of_objs.first

  csv = FasterCSV.new(io, options)

  # write meta information
  begin
    csv << obj_template.class.csv_meta
  rescue NoMethodError
    csv << [:class, obj_template.class]
  end

  # write headers
  begin
    headers = obj_template.csv_headers
  rescue NoMethodError
    headers = obj_template.instance_variables.sort
    if obj_template.class.ancestors.find { |cls| cls.to_s =~ /\AStruct\b/ }
      headers += obj_template.members.map { |mem| "#{mem}=" }.sort
    end
  end
  csv << headers

  # serialize each object
  ary_of_objs.each do |obj|
    begin
      csv << obj.csv_dump(headers)
    rescue NoMethodError
      csv << headers.map do |var|
        if var[0] == ?@
          obj.instance_variable_get(var)
        else
          obj[var[0..-2]]
        end
      end
    end
  end

  if io.is_a? String
    csv.string
  else
    csv.close
  end
end

.filter(*args) ⇒ Object

:call-seq:

filter( options = Hash.new ) { |row| ... }
filter( input, options = Hash.new ) { |row| ... }
filter( input, output, options = Hash.new ) { |row| ... }

This method is a convenience for building Unix-like filters for CSV data. Each row is yielded to the provided block which can alter it as needed.

After the block returns, the row is appended to output altered or not.

The input and output arguments can be anything FasterCSV::new() accepts (generally String or IO objects). If not given, they default to ARGF and $stdout.

The options parameter is also filtered down to FasterCSV::new() after some clever key parsing. Any key beginning with :in_ or :input_ will have that leading identifier stripped and will only be used in the options Hash for the input object. Keys starting with :out_ or :output_ affect only output. All other keys are assigned to both objects.

The :output_row_sep option defaults to $INPUT_RECORD_SEPARATOR ($/).



1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
 
# File 'lib/faster_csv.rb', line 1012

def self.filter(*args)
  # parse options for input, output, or both
  in_options, out_options = Hash.new, {:row_sep => $INPUT_RECORD_SEPARATOR}
  if args.last.is_a? Hash
    args.pop.each do |key, value|
      case key.to_s
      when /\Ain(?:put)?_(.+)\Z/
        in_options[$1.to_sym] = value
      when /\Aout(?:put)?_(.+)\Z/
        out_options[$1.to_sym] = value
      else
        in_options[key]  = value
        out_options[key] = value
      end
    end
  end
  # build input and output wrappers
  input   = FasterCSV.new(args.shift || ARGF,    in_options)
  output  = FasterCSV.new(args.shift || $stdout, out_options)

  # read, yield, write
  input.each do |row|
    yield row
    output << row
  end
end

.foreach(path, options = Hash.new, &block) ⇒ Object

This method is intended as the primary interface for reading CSV files. You pass a path and any options you wish to set for the read. Each row of file will be passed to the provided block in turn.

The options parameter can be anything FasterCSV::new() understands.



1046
1047
1048
1049
1050
 
# File 'lib/faster_csv.rb', line 1046

def self.foreach(path, options = Hash.new, &block)
  open(path, "rb", options) do |csv|
    csv.each(&block)
  end
end

.generate(*args) {|faster_csv| ... } ⇒ Object

:call-seq:

generate( str, options = Hash.new ) { |faster_csv| ... }
generate( options = Hash.new ) { |faster_csv| ... }

This method wraps a String you provide, or an empty default String, in a FasterCSV object which is passed to the provided block. You can use the block to append CSV rows to the String and when the block exits, the final String will be returned.

Note that a passed String is modfied by this method. Call dup() before passing if you need a new String.

The options parameter can be anthing FasterCSV::new() understands.

Yields:

  • (faster_csv) 


1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1079
 
# File 'lib/faster_csv.rb', line 1067

def self.generate(*args)
  # add a default empty String, if none was given
  if args.first.is_a? String
    io = StringIO.new(args.shift)
    io.seek(0, IO::SEEK_END)
    args.unshift(io)
  else
    args.unshift("")
  end
  faster_csv = new(*args)  # wrap
  yield faster_csv         # yield for appending
  faster_csv.string        # return final String
end

.generate_line(row, options = Hash.new) ⇒ Object

This method is a shortcut for converting a single row (Array) into a CSV String.

The options parameter can be anthing FasterCSV::new() understands.

The :row_sep option defaults to $INPUT_RECORD_SEPARATOR ($/) when calling this method.



1090
1091
1092
1093
 
# File 'lib/faster_csv.rb', line 1090

def self.generate_line(row, options = Hash.new)
  options = {:row_sep => $INPUT_RECORD_SEPARATOR}.merge(options)
  (new("", options) << row).string
end

.instance(data = $stdout, options = Hash.new) ⇒ Object

This method will return a FasterCSV instance, just like FasterCSV::new(), but the instance will be cached and returned for all future calls to this method for the same data object (tested by Object#object_id()) with the same options.

If a block is given, the instance is passed to the block and the return value becomes the return value of the block.



1104
1105
1106
1107
1108
1109
1110
1111
1112
1113
1114
1115
1116
1117
1118
 
# File 'lib/faster_csv.rb', line 1104

def self.instance(data = $stdout, options = Hash.new)
  # create a _signature_ for this method call, data object and options
  sig = [data.object_id] +
        options.values_at(*DEFAULT_OPTIONS.keys.sort_by { |sym| sym.to_s })

  # fetch or create the instance for this signature
  @@instances ||= Hash.new
  instance    =   (@@instances[sig] ||= new(data, options))

  if block_given?
    yield instance  # run block, if given, returning result
  else
    instance        # or return the instance
  end
end

.load(io_or_str, options = Hash.new) ⇒ Object

This method is the reading counterpart to FasterCSV::dump(). See that method for a detailed description of the process.

You can customize loading by adding a class method called csv_load() which will be passed a Hash of meta information, an Array of headers, and an Array of fields for the object the method is expected to return.

Remember that all fields will be Strings after this load. If you need something else, use options to setup converters or provide a custom csv_load() implementation.



1132
1133
1134
1135
1136
1137
1138
1139
1140
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158
1159
1160
1161
1162
1163
1164
 
# File 'lib/faster_csv.rb', line 1132

def self.load(io_or_str, options = Hash.new)
  csv = FasterCSV.new(io_or_str, options)

  # load meta information
  meta = Hash[*csv.shift]
  cls  = meta["class"].split("::").inject(Object) do |c, const|
    c.const_get(const)
  end

  # load headers
  headers = csv.shift

  # unserialize each object stored in the file
  results = csv.inject(Array.new) do |all, row|
    begin
      obj = cls.csv_load(meta, headers, row)
    rescue NoMethodError
      obj = cls.allocate
      headers.zip(row) do |name, value|
        if name[0] == ?@
          obj.instance_variable_set(name, value)
        else
          obj.send(name, value)
        end
      end
    end
    all << obj
  end

  csv.close unless io_or_str.is_a? String

  results
end

.method_missing(*_) ⇒ Object 



18
19
20
 
# File 'lib/faster_csv.rb', line 18

def self.method_missing(*_)
  const_missing
end

.open(*args) ⇒ Object

:call-seq:

open( filename, mode="rb", options = Hash.new ) { |faster_csv| ... }
open( filename, mode="rb", options = Hash.new )

This method opens an IO object, and wraps that with FasterCSV. This is intended as the primary interface for writing a CSV file.

You may pass any args Ruby’s open() understands followed by an optional Hash containing any options FasterCSV::new() understands.

This method works like Ruby’s open() call, in that it will pass a FasterCSV object to a provided block and close it when the block termminates, or it will return the FasterCSV object when no block is provided. (Note: This is different from the standard CSV library which passes rows to the block.

Use FasterCSV::foreach() for that behavior.)

An opened FasterCSV object will delegate to many IO methods, for convenience. You may call:

  • binmode()

  • close()

  • close_read()

  • close_write()

  • closed?()

  • eof()

  • eof?()

  • fcntl()

  • fileno()

  • flush()

  • fsync()

  • ioctl()

  • isatty()

  • pid()

  • pos()

  • reopen()

  • seek()

  • stat()

  • sync()

  • sync=()

  • tell()

  • to_i()

  • to_io()

  • tty?()



1211
1212
1213
1214
1215
1216
1217
1218
1219
1220
1221
1222
1223
1224
1225
1226
1227
1228
1229
 
# File 'lib/faster_csv.rb', line 1211

def self.open(*args)
  # find the +options+ Hash
  options = if args.last.is_a? Hash then args.pop else Hash.new end
  # default to a binary open mode
  args << "rb" if args.size == 1
  # wrap a File opened with the remaining +args+
  csv     = new(File.open(*args), options)

  # handle blocks like Ruby's open(), not like the CSV library
  if block_given?
    begin
      yield csv
    ensure
      csv.close
    end
  else
    csv
  end
end

.parse(*args, &block) ⇒ Object

:call-seq:

parse( str, options = Hash.new ) { |row| ... }
parse( str, options = Hash.new )

This method can be used to easily parse CSV out of a String. You may either provide a block which will be called with each row of the String in turn, or just use the returned Array of Arrays (when no block is given).

You pass your str to read from, and an optional options Hash containing anything FasterCSV::new() understands.



1243
1244
1245
1246
1247
1248
1249
1250
1251
1252
1253
1254
 
# File 'lib/faster_csv.rb', line 1243

def self.parse(*args, &block)
  csv = new(*args)
  if block.nil?  # slurp contents, if no block is given
    begin
      csv.read
    ensure
      csv.close
    end
  else           # or pass each row to a provided block
    csv.each(&block)
  end
end

.parse_line(line, options = Hash.new) ⇒ Object

This method is a shortcut for converting a single line of a CSV String into a into an Array. Note that if line contains multiple rows, anything beyond the first row is ignored.

The options parameter can be anything FasterCSV::new() understands.



1263
1264
1265
 
# File 'lib/faster_csv.rb', line 1263

def self.parse_line(line, options = Hash.new)
  new(line, options).shift
end

.read(path, options = Hash.new) ⇒ Object

Use to slurp a CSV file into an Array of Arrays. Pass the path to the file and any options FasterCSV::new() understands.



1271
1272
1273
 
# File 'lib/faster_csv.rb', line 1271

def self.read(path, options = Hash.new)
  open(path, "rb", options) { |csv| csv.read }
end

.readlines(*args) ⇒ Object

Alias for FasterCSV::read().



1276
1277
1278
 
# File 'lib/faster_csv.rb', line 1276

def self.readlines(*args)
  read(*args)
end

.table(path, options = Hash.new) ⇒ Object

A shortcut for:

FasterCSV.read( path, { :headers           => true,
                        :converters        => :numeric,
                        :header_converters => :symbol }.merge(options) )



1287
1288
1289
1290
1291
 
# File 'lib/faster_csv.rb', line 1287

def self.table(path, options = Hash.new)
  read( path, { :headers           => true,
                :converters        => :numeric,
                :header_converters => :symbol }.merge(options) )
end

Instance Method Details

#<<(row) ⇒ Object Also known as: add_row, puts

The primary write method for wrapped Strings and IOs, row (an Array or FasterCSV::Row) is converted to CSV and appended to the data source. When a FasterCSV::Row is passed, only the row’s fields() are appended to the output.

The data source must be open for writing.



1475
1476
1477
1478
1479
1480
1481
1482
1483
1484
1485
1486
1487
1488
1489
1490
1491
1492
1493
1494
1495
 
# File 'lib/faster_csv.rb', line 1475

def <<(row)
  # make sure headers have been assigned
  if header_row? and [Array, String].include? @use_headers.class
    parse_headers  # won't read data for Array or String
    self << @headers if @write_headers
  end

  # Handle FasterCSV::Row objects and Hashes
  row = case row
        when self.class::Row then row.fields
        when Hash            then @headers.map { |header| row[header] }
        else                      row
        end

  @headers =  row if header_row?
  @lineno  += 1

  @io << row.map(&@quote).join(@col_sep) + @row_sep  # quote and separate

  self  # for chaining
end

#convert(name = nil, &converter) ⇒ Object

:call-seq:

convert( name )
convert { |field| ... }
convert { |field, field_info| ... }

You can use this method to install a FasterCSV::Converters built-in, or provide a block that handles a custom conversion.

If you provide a block that takes one argument, it will be passed the field and is expected to return the converted value or the field itself. If your block takes two arguments, it will also be passed a FieldInfo Struct, containing details about the field. Again, the block should return a converted field or the field itself.



1514
1515
1516
 
# File 'lib/faster_csv.rb', line 1514

def convert(name = nil, &converter)
  add_converter(:converters, self.class::Converters, name, &converter)
end

#each ⇒ Object

Yields each row of the data source in turn.

Support for Enumerable.

The data source must be open for reading.



1545
1546
1547
1548
1549
 
# File 'lib/faster_csv.rb', line 1545

def each
  while row = shift
    yield row
  end
end

#header_convert(name = nil, &converter) ⇒ Object

:call-seq:

header_convert( name )
header_convert { |field| ... }
header_convert { |field, field_info| ... }

Identical to FasterCSV.convert(), but for header rows.

Note that this method must be called before header rows are read to have any effect.



1529
1530
1531
1532
1533
1534
 
# File 'lib/faster_csv.rb', line 1529

def header_convert(name = nil, &converter)
  add_converter( :header_converters,
                 self.class::HeaderConverters,
                 name,
                 &converter )
end

#header_row? ⇒ Boolean

Returns true if the next row read will be a header row.

Returns:

  • (Boolean) 


1567
1568
1569
 
# File 'lib/faster_csv.rb', line 1567

def header_row?
  @use_headers and @headers.nil?
end

#inspect ⇒ Object

Returns a simplified description of the key FasterCSV attributes.



1696
1697
1698
1699
1700
1701
1702
1703
1704
1705
1706
1707
1708
1709
1710
1711
1712
1713
1714
1715
1716
1717
1718
1719
 
# File 'lib/faster_csv.rb', line 1696

def inspect
  str = "<##{self.class} io_type:"
  # show type of wrapped IO
  if    @io == $stdout then str << "$stdout"
  elsif @io == $stdin  then str << "$stdin"
  elsif @io == $stderr then str << "$stderr"
  else                      str << @io.class.to_s
  end
  # show IO.path(), if available
  if @io.respond_to?(:path) and (p = @io.path)
    str << " io_path:#{p.inspect}"
  end
  # show other attributes
  %w[ lineno     col_sep     row_sep
      quote_char skip_blanks encoding ].each do |attr_name|
    if a = instance_variable_get("@#{attr_name}")
      str << " #{attr_name}:#{a.inspect}"
    end
  end
  if @use_headers
    str << " headers:#{(@headers || true).inspect}"
  end
  str << ">"
end

#read ⇒ Object Also known as: readlines

Slurps the remaining rows and returns an Array of Arrays.

The data source must be open for reading.



1556
1557
1558
1559
1560
1561
1562
1563
 
# File 'lib/faster_csv.rb', line 1556

def read
  rows = to_a
  if @use_headers
    Table.new(rows)
  else
    rows
  end
end

#rewind ⇒ Object

Rewinds the underlying IO object and resets FasterCSV’s lineno() counter.



1458
1459
1460
1461
1462
1463
 
# File 'lib/faster_csv.rb', line 1458

def rewind
  @headers = nil
  @lineno  = 0

  @io.rewind
end

#shift ⇒ Object Also known as: gets, readline

The primary read method for wrapped Strings and IOs, a single row is pulled from the data source, parsed and returned as an Array of fields (if header rows are not used) or a FasterCSV::Row (when header rows are used).

The data source must be open for reading.



1578
1579
1580
1581
1582
1583
1584
1585
1586
1587
1588
1589
1590
1591
1592
1593
1594
1595
1596
1597
1598
1599
1600
1601
1602
1603
1604
1605
1606
1607
1608
1609
1610
1611
1612
1613
1614
1615
1616
1617
1618
1619
1620
1621
1622
1623
1624
1625
1626
1627
1628
1629
1630
1631
1632
1633
1634
1635
1636
1637
1638
1639
1640
1641
1642
1643
1644
1645
1646
1647
1648
1649
1650
1651
1652
1653
1654
1655
1656
1657
1658
1659
1660
1661
1662
1663
1664
1665
1666
1667
1668
1669
1670
1671
1672
1673
1674
1675
1676
1677
1678
1679
1680
1681
1682
1683
1684
1685
1686
1687
1688
1689
1690
1691
 
# File 'lib/faster_csv.rb', line 1578

def shift
  #########################################################################
  ### This method is purposefully kept a bit long as simple conditional ###
  ### checks are faster than numerous (expensive) method calls.         ###
  #########################################################################

  # handle headers not based on document content
  if header_row? and @return_headers and
     [Array, String].include? @use_headers.class
    if @unconverted_fields
      return add_unconverted_fields(parse_headers, Array.new)
    else
      return parse_headers
    end
  end

  # begin with a blank line, so we can always add to it
  line = String.new

  # 
  # it can take multiple calls to <tt>@io.gets()</tt> to get a full line,
  # because of \r and/or \n characters embedded in quoted fields
  # 
  loop do
    # add another read to the line
    if read_line = @io.gets(@row_sep)
     line += read_line
    else
     return nil
    end
    # copy the line so we can chop it up in parsing
    parse =  line.dup
    parse.sub!(@parsers[:line_end], "")

    # 
    # I believe a blank line should be an <tt>Array.new</tt>, not 
    # CSV's <tt>[nil]</tt>
    # 
    if parse.empty?
      @lineno += 1
      if @skip_blanks
        line = ""
        next
      elsif @unconverted_fields
        return add_unconverted_fields(Array.new, Array.new)
      elsif @use_headers
        return FasterCSV::Row.new(Array.new, Array.new)
      else
        return Array.new
      end
    end

    # parse the fields with a mix of String#split and regular expressions
    csv           = Array.new
    current_field = String.new
    field_quotes  = 0
    parse.split(@col_sep, -1).each do |match|
      if current_field.empty? && match.count(@quote_and_newlines).zero?
        csv           << (match.empty? ? nil : match)
      elsif (current_field.empty? ? match[0] : current_field[0]) ==
            @quote_char[0]
        current_field << match
        field_quotes += match.count(@quote_char)
        if field_quotes % 2 == 0
          in_quotes = current_field[@parsers[:quoted_field], 1]
          if !in_quotes || in_quotes[@parsers[:stray_quote]]
            raise MalformedCSVError,
                  "Missing or stray quote in line #{lineno + 1}"
          end
          current_field = in_quotes
          current_field.gsub!(@quote_char * 2, @quote_char) # unescape contents
          csv           << current_field
          current_field =  String.new
          field_quotes  =  0
        else # we found a quoted field that spans multiple lines
          current_field << @col_sep
        end
      elsif match.count("\r\n").zero?
        raise MalformedCSVError, "Illegal quoting in line #{lineno + 1}."
      else
        raise MalformedCSVError, "Unquoted fields do not allow " +
                                 "\\r or \\n (line #{lineno + 1})."
      end
    end

    # if parse is empty?(), we found all the fields on the line...
    if field_quotes % 2 == 0
      @lineno += 1

      # save fields unconverted fields, if needed...
      unconverted = csv.dup if @unconverted_fields

      # convert fields, if needed...
      csv = convert_fields(csv) unless @use_headers or @converters.empty?
      # parse out header rows and handle FasterCSV::Row conversions...
      csv = parse_headers(csv)  if     @use_headers

      # inject unconverted fields and accessor, if requested...
      if @unconverted_fields and not csv.respond_to? :unconverted_fields
        add_unconverted_fields(csv, unconverted)
      end

      # return the results
      break csv
    end
    # if we're not empty?() but at eof?(), a quoted field wasn't closed...
    if @io.eof?
      raise MalformedCSVError, "Unclosed quoted field on line #{lineno + 1}."
    elsif @field_size_limit and current_field.size >= @field_size_limit
      raise MalformedCSVError, "Field size exceeded on line #{lineno + 1}."
    end
    # otherwise, we need to loop and pull some more data to complete the row
  end
end