Class: CSV
- Inherits:
-
Object
- Object
- CSV
- Extended by:
- Forwardable
- Includes:
- Enumerable
- Defined in:
- lib/csv.rb
Overview
This class provides a complete interface to CSV files and data. It offers tools to enable you to read and write to and from Strings or IO objects, as needed.
Reading
From a File
A Line at a Time
CSV.foreach("path/to/file.csv") do |row|
# use row here...
end
All at Once
arr_of_arrs = CSV.read("path/to/file.csv")
From a String
A Line at a Time
CSV.parse("CSV,data,String") do |row|
# use row here...
end
All at Once
arr_of_arrs = CSV.parse("CSV,data,String")
Writing
To a File
CSV.open("path/to/file.csv", "wb") do |csv|
csv << ["row", "of", "CSV", "data"]
csv << ["another", "row"]
# ...
end
To a String
csv_string = CSV.generate do |csv|
csv << ["row", "of", "CSV", "data"]
csv << ["another", "row"]
# ...
end
Convert a Single Line
csv_string = ["CSV", "data"].to_csv # to CSV
csv_array = "CSV,String".parse_csv # from CSV
Shortcut Interface
CSV { |csv_out| csv_out << %w{my data here} } # to $stdout
CSV(csv = "") { |csv_str| csv_str << %w{my data here} } # to a String
CSV($stderr) { |csv_err| csv_err << %w{my data here} } # to $stderr
CSV($stdin) { |csv_in| csv_in.each { |row| p row } } # from $stdin
Advanced Usage
Wrap an IO Object
csv = CSV.new(io, )
# ... read (with gets() or each()) from and write (with <<) to csv here ...
CSV and Character Encodings (M17n or Multilingualization)
This new CSV parser is m17n savvy. The parser works in the Encoding of the IO or String object being read from or written to. Your data is never transcoded (unless you ask Ruby to transcode it for you) and will literally be parsed in the Encoding it is in. Thus CSV will return Arrays or Rows of Strings in the Encoding of your data. This is accomplished by transcoding the parser itself into your Encoding.
Some transcoding must take place, of course, to accomplish this multiencoding support. For example, :col_sep
, :row_sep
, and :quote_char
must be transcoded to match your data. Hopefully this makes the entire process feel transparent, since CSV’s defaults should just magically work for your data. However, you can set these values manually in the target Encoding to avoid the translation.
It’s also important to note that while all of CSV’s core parser is now Encoding agnostic, some features are not. For example, the built-in converters will try to transcode data to UTF-8 before making conversions. Again, you can provide custom converters that are aware of your Encodings to avoid this translation. It’s just too hard for me to support native conversions in all of Ruby’s Encodings.
Anyway, the practical side of this is simple: make sure IO and String objects passed into CSV have the proper Encoding set and everything should just work. CSV methods that allow you to open IO objects (CSV::foreach(), CSV::open(), CSV::read(), and CSV::readlines()) do allow you to specify the Encoding.
One minor exception comes when generating CSV into a String with an Encoding that is not ASCII compatible. There’s no existing data for CSV to use to prepare itself and thus you will probably need to manually specify the desired Encoding for most of those cases. It will try to guess using the fields in a row of output though, when using CSV::generate_line() or Array#to_csv().
I try to point out any other Encoding issues in the documentation of methods as they come up.
This has been tested to the best of my ability with all non-“dummy” Encodings Ruby ships with. However, it is brave new code and may have some bugs. Please feel free to report any issues you find with it.
Defined Under Namespace
Classes: FieldInfo, MalformedCSVError, Row, Table
Constant Summary collapse
- VERSION =
The version of the installed library.
"1.0.1"
- DateMatcher =
A Regexp used to find and convert some common Date formats.
/ \A(?: (\w+,?\s+)?\w+\s+\d{1,2},?\s+\d{2,4} | \d{4}-\d{2}-\d{2} )\z /x
- DateTimeMatcher =
A Regexp used to find and convert some common DateTime formats.
/ \A(?: (\w+,?\s+)?\w+\s+\d{1,2}\s+\d{1,2}:\d{1,2}:\d{1,2},?\s+\d{2,4} | \d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2} | # ISO-8601 \d{4}-\d{2}-\d{2} (?:T\d{2}:\d{2}(?::\d{2}(?:\.\d+)?(?:[+-]\d{2}(?::\d{2})|Z)?)?)? )\z /x
- ConverterEncoding =
The encoding used by all converters.
Encoding.find("UTF-8")
- Converters =
This Hash holds the built-in converters of CSV that can be accessed by name. You can select Converters with CSV.convert() or through the
options
Hash passed to CSV::new().:integer
-
Converts any field Integer() accepts.
:float
-
Converts any field Float() accepts.
:numeric
-
A combination of
:integer
and:float
. :date
-
Converts any field Date::parse() accepts.
:date_time
-
Converts any field DateTime::parse() accepts.
:all
-
All built-in converters. A combination of
:date_time
and:numeric
.
All built-in converters transcode field data to UTF-8 before attempting a conversion. If your data cannot be transcoded to UTF-8 the conversion will fail and the field will remain unchanged.
This Hash is intentionally left unfrozen and users should feel free to add values to it that can be accessed by all CSV objects.
To add a combo field, the value should be an Array of names. Combo fields can be nested with other combo fields.
{ integer: lambda { |f| Integer(f.encode(ConverterEncoding)) rescue f }, float: lambda { |f| Float(f.encode(ConverterEncoding)) rescue f }, numeric: [:integer, :float], date: lambda { |f| begin e = f.encode(ConverterEncoding) e =~ DateMatcher ? Date.parse(e) : f rescue # encoding conversion or date parse errors f end }, date_time: lambda { |f| begin e = f.encode(ConverterEncoding) e =~ DateTimeMatcher ? DateTime.parse(e) : f rescue # encoding conversion or date parse errors f end }, all: [:date_time, :numeric], }
- HeaderConverters =
This Hash holds the built-in header converters of CSV that can be accessed by name. You can select HeaderConverters with CSV.header_convert() or through the
options
Hash passed to CSV::new().:downcase
-
Calls downcase() on the header String.
:symbol
-
Leading/trailing spaces are dropped, string is downcased, remaining spaces are replaced with underscores, non-word characters are dropped, and finally to_sym() is called.
All built-in header converters transcode header data to UTF-8 before attempting a conversion. If your data cannot be transcoded to UTF-8 the conversion will fail and the header will remain unchanged.
This Hash is intentionally left unfrozen and users should feel free to add values to it that can be accessed by all CSV objects.
To add a combo field, the value should be an Array of names. Combo fields can be nested with other combo fields.
{ downcase: lambda { |h| h.encode(ConverterEncoding).downcase }, symbol: lambda { |h| h.encode(ConverterEncoding).downcase.gsub(/[^\s\w]+/, "").strip. gsub(/\s+/, "_").to_sym } }
- DEFAULT_OPTIONS =
The options used when no overrides are given by calling code. They are:
:col_sep
-
","
:row_sep
-
:auto
:quote_char
-
'"'
:field_size_limit
-
nil
:converters
-
nil
:unconverted_fields
-
nil
:headers
-
false
:return_headers
-
false
:header_converters
-
nil
:skip_blanks
-
false
:force_quotes
-
false
:skip_lines
-
nil
:liberal_parsing
-
false
{ col_sep: ",", row_sep: :auto, quote_char: '"', field_size_limit: nil, converters: nil, unconverted_fields: nil, headers: false, return_headers: false, header_converters: nil, skip_blanks: false, force_quotes: false, skip_lines: nil, liberal_parsing: false, }.freeze
Instance Attribute Summary collapse
-
#col_sep ⇒ Object
readonly
The encoded
:col_sep
used in parsing and writing. -
#encoding ⇒ Object
readonly
The Encoding CSV is parsing or writing in.
-
#field_size_limit ⇒ Object
readonly
The limit for field size, if any.
-
#line ⇒ Object
readonly
The line number of the last row read from this file.
-
#lineno ⇒ Object
readonly
The line number of the last row read from this file.
-
#quote_char ⇒ Object
readonly
The encoded
:quote_char
used in parsing and writing. -
#row_sep ⇒ Object
readonly
The encoded
:row_sep
used in parsing and writing. -
#skip_lines ⇒ Object
readonly
The regex marking a line as a comment.
Class Method Summary collapse
-
.filter(input = nil, output = nil, **options) ⇒ Object
:call-seq: filter( **options ) { |row| … } filter( input, **options ) { |row| … } filter( input, output, **options ) { |row| … }.
-
.foreach(path, **options, &block) ⇒ Object
This method is intended as the primary interface for reading CSV files.
-
.generate(str = nil, **options) {|csv| ... } ⇒ Object
:call-seq: generate( str, **options ) { |csv| … } generate( **options ) { |csv| … }.
-
.generate_line(row, **options) ⇒ Object
This method is a shortcut for converting a single row (Array) into a CSV String.
-
.instance(data = $stdout, **options) ⇒ Object
This method will return a CSV instance, just like CSV::new(), but the instance will be cached and returned for all future calls to this method for the same
data
object (tested by Object#object_id()) with the sameoptions
. -
.open(filename, mode = "r", **options) ⇒ Object
:call-seq: open( filename, mode = “rb”, **options ) { |faster_csv| … } open( filename, **options ) { |faster_csv| … } open( filename, mode = “rb”, **options ) open( filename, **options ).
-
.parse(*args, &block) ⇒ Object
:call-seq: parse( str, **options ) { |row| … } parse( str, **options ).
-
.parse_line(line, **options) ⇒ Object
This method is a shortcut for converting a single line of a CSV String into an Array.
-
.read(path, *options) ⇒ Object
Use to slurp a CSV file into an Array of Arrays.
-
.readlines(*args) ⇒ Object
Alias for CSV::read().
-
.table(path, **options) ⇒ Object
A shortcut for:.
Instance Method Summary collapse
-
#<<(row) ⇒ Object
(also: #add_row, #puts)
The primary write method for wrapped Strings and IOs,
row
(an Array or CSV::Row) is converted to CSV and appended to the data source. -
#convert(name = nil, &converter) ⇒ Object
:call-seq: convert( name ) convert { |field| … } convert { |field, field_info| … }.
-
#converters ⇒ Object
Returns the current list of converters in effect.
-
#each ⇒ Object
Yields each row of the data source in turn.
-
#force_quotes? ⇒ Boolean
Returns
true
if all output fields are quoted. -
#header_convert(name = nil, &converter) ⇒ Object
:call-seq: header_convert( name ) header_convert { |field| … } header_convert { |field, field_info| … }.
-
#header_converters ⇒ Object
Returns the current list of converters in effect for headers.
-
#header_row? ⇒ Boolean
Returns
true
if the next row read will be a header row. -
#headers ⇒ Object
Returns
nil
if headers will not be used,true
if they will but have not yet been read, or the actual headers after they have been read. -
#initialize(data, col_sep: ",", row_sep: :auto, quote_char: '"', field_size_limit: nil, converters: nil, unconverted_fields: nil, headers: false, return_headers: false, write_headers: nil, header_converters: nil, skip_blanks: false, force_quotes: false, skip_lines: nil, liberal_parsing: false, internal_encoding: nil, external_encoding: nil, encoding: nil) ⇒ CSV
constructor
This constructor will wrap either a String or IO object passed in
data
for reading and/or writing. -
#inspect ⇒ Object
Returns a simplified description of the key CSV attributes in an ASCII compatible String.
-
#liberal_parsing? ⇒ Boolean
Returns
true
if illegal input is handled. -
#read ⇒ Object
(also: #readlines)
Slurps the remaining rows and returns an Array of Arrays.
-
#return_headers? ⇒ Boolean
Returns
true
if headers will be returned as a row of results. -
#rewind ⇒ Object
Rewinds the underlying IO object and resets CSV’s lineno() counter.
-
#shift ⇒ Object
(also: #gets, #readline)
The primary read method for wrapped Strings and IOs, a single row is pulled from the data source, parsed and returned as an Array of fields (if header rows are not used) or a CSV::Row (when header rows are used).
-
#skip_blanks? ⇒ Boolean
Returns
true
blank lines are skipped by the parser. -
#unconverted_fields? ⇒ Boolean
Returns
true
if unconverted_fields() to parsed results. -
#write_headers? ⇒ Boolean
Returns
true
if headers are written in output.
Constructor Details
#initialize(data, col_sep: ",", row_sep: :auto, quote_char: '"', field_size_limit: nil, converters: nil, unconverted_fields: nil, headers: false, return_headers: false, write_headers: nil, header_converters: nil, skip_blanks: false, force_quotes: false, skip_lines: nil, liberal_parsing: false, internal_encoding: nil, external_encoding: nil, encoding: nil) ⇒ CSV
This constructor will wrap either a String or IO object passed in data
for reading and/or writing. In addition to the CSV instance methods, several IO methods are delegated. (See CSV::open() for a complete list.) If you pass a String for data
, you can later retrieve it (after writing to it, for example) with CSV.string().
Note that a wrapped String will be positioned at the beginning (for reading). If you want it at the end (for writing), use CSV::generate(). If you want any other positioning, pass a preset StringIO object instead.
You may set any reading and/or writing preferences in the options
Hash. Available options are:
:col_sep
-
The String placed between each field. This String will be transcoded into the data’s Encoding before parsing.
:row_sep
-
The String appended to the end of each row. This can be set to the special
:auto
setting, which requests that CSV automatically discover this from the data. Auto-discovery reads ahead in the data looking for the next"\r\n"
,"\n"
, or"\r"
sequence. A sequence will be selected even if it occurs in a quoted field, assuming that you would have the same line endings there. If none of those sequences is found,data
isARGF
,STDIN
,STDOUT
, orSTDERR
, or the stream is only available for output, the default$INPUT_RECORD_SEPARATOR
($/
) is used. Obviously, discovery takes a little time. Set manually if speed is important. Also note that IO objects should be opened in binary mode on Windows if this feature will be used as the line-ending translation can cause problems with resetting the document position to where it was before the read ahead. This String will be transcoded into the data’s Encoding before parsing. :quote_char
-
The character used to quote fields. This has to be a single character String. This is useful for application that incorrectly use
'
as the quote character instead of the correct"
. CSV will always consider a double sequence of this character to be an escaped quote. This String will be transcoded into the data’s Encoding before parsing. :field_size_limit
-
This is a maximum size CSV will read ahead looking for the closing quote for a field. (In truth, it reads to the first line ending beyond this size.) If a quote cannot be found within the limit CSV will raise a MalformedCSVError, assuming the data is faulty. You can use this limit to prevent what are effectively DoS attacks on the parser. However, this limit can cause a legitimate parse to fail and thus is set to
nil
, or off, by default. :converters
-
An Array of names from the Converters Hash and/or lambdas that handle custom conversion. A single converter doesn’t have to be in an Array. All built-in converters try to transcode fields to UTF-8 before converting. The conversion will fail if the data cannot be transcoded, leaving the field unchanged.
:unconverted_fields
-
If set to
true
, an unconverted_fields() method will be added to all returned rows (Array or CSV::Row) that will return the fields as they were before conversion. Note that:headers
supplied by Array or String were not fields of the document and thus will have an empty Array attached. :headers
-
If set to
:first_row
ortrue
, the initial row of the CSV file will be treated as a row of headers. If set to an Array, the contents will be used as the headers. If set to a String, the String is run through a call of CSV::parse_line() with the same:col_sep
,:row_sep
, and:quote_char
as this instance to produce an Array of headers. This setting causes CSV#shift() to return rows as CSV::Row objects instead of Arrays and CSV#read() to return CSV::Table objects instead of an Array of Arrays. :return_headers
-
When
false
, header rows are silently swallowed. If set totrue
, header rows are returned in a CSV::Row object with identical headers and fields (save that the fields do not go through the converters). :write_headers
-
When
true
and:headers
is set, a header row will be added to the output. :header_converters
-
Identical in functionality to
:converters
save that the conversions are only made to header rows. All built-in converters try to transcode headers to UTF-8 before converting. The conversion will fail if the data cannot be transcoded, leaving the header unchanged. :skip_blanks
-
When set to a
true
value, CSV will skip over any empty rows. Note that this setting will not skip rows that contain column separators, even if the rows contain no actual data. If you want to skip rows that contain separators but no content, consider using:skip_lines
, or inspecting fields.compact.empty? on each row. :force_quotes
-
When set to a
true
value, CSV will quote all CSV fields it creates. :skip_lines
-
When set to an object responding to
match
, every line matching it is considered a comment and ignored during parsing. When set to a String, it is first converted to a Regexp. When set tonil
no line is considered a comment. If the passed object does not respond tomatch
,ArgumentError
is thrown. :liberal_parsing
-
When set to a
true
value, CSV will attempt to parse input not conformant with RFC 4180, such as double quotes in unquoted fields.
See CSV::DEFAULT_OPTIONS for the default settings.
Options cannot be overridden in the instance methods for performance reasons, so be sure to set what you want here.
1569 1570 1571 1572 1573 1574 1575 1576 1577 1578 1579 1580 1581 1582 1583 1584 1585 1586 1587 1588 1589 1590 1591 1592 1593 1594 1595 1596 1597 1598 1599 1600 1601 1602 1603 1604 1605 1606 1607 1608 1609 1610 1611 1612 1613 1614 1615 1616 1617 1618 |
# File 'lib/csv.rb', line 1569 def initialize(data, col_sep: ",", row_sep: :auto, quote_char: '"', field_size_limit: nil, converters: nil, unconverted_fields: nil, headers: false, return_headers: false, write_headers: nil, header_converters: nil, skip_blanks: false, force_quotes: false, skip_lines: nil, liberal_parsing: false, internal_encoding: nil, external_encoding: nil, encoding: nil) raise ArgumentError.new("Cannot parse nil as CSV") if data.nil? # create the IO object we will read from @io = data.is_a?(String) ? StringIO.new(data) : data # honor the IO encoding if we can, otherwise default to ASCII-8BIT internal_encoding = Encoding.find(internal_encoding) if internal_encoding external_encoding = Encoding.find(external_encoding) if external_encoding if encoding encoding, = encoding.split(":", 2) if encoding.is_a?(String) encoding = Encoding.find(encoding) end @encoding = raw_encoding(nil) || internal_encoding || encoding || Encoding.default_internal || Encoding.default_external # # prepare for building safe regular expressions in the target encoding, # if we can transcode the needed characters # @re_esc = "\\".encode(@encoding).freeze rescue "" @re_chars = /#{%"[-\\]\\[\\.^$?*+{}()|# \r\n\t\f\v]".encode(@encoding)}/ @unconverted_fields = unconverted_fields # Stores header row settings and loads header converters, if needed. @use_headers = headers @return_headers = return_headers @write_headers = write_headers # headers must be delayed until shift(), in case they need a row of content @headers = nil init_separators(col_sep, row_sep, quote_char, force_quotes) init_parsers(skip_blanks, field_size_limit, liberal_parsing) init_converters(converters, :@converters, :convert) init_converters(header_converters, :@header_converters, :header_convert) init_comments(skip_lines) @force_encoding = !!encoding # track our own lineno since IO gets confused about line-ends is CSV fields @lineno = 0 # make sure headers have been assigned if header_row? and [Array, String].include? @use_headers.class and @write_headers parse_headers # won't read data for Array or String self << @headers end end |
Instance Attribute Details
#col_sep ⇒ Object (readonly)
The encoded :col_sep
used in parsing and writing. See CSV::new for details.
1624 1625 1626 |
# File 'lib/csv.rb', line 1624 def col_sep @col_sep end |
#encoding ⇒ Object (readonly)
The Encoding CSV is parsing or writing in. This will be the Encoding you receive parsed data in and/or the Encoding data will be written in.
1697 1698 1699 |
# File 'lib/csv.rb', line 1697 def encoding @encoding end |
#field_size_limit ⇒ Object (readonly)
The limit for field size, if any. See CSV::new for details.
1636 1637 1638 |
# File 'lib/csv.rb', line 1636 def field_size_limit @field_size_limit end |
#line ⇒ Object (readonly)
The line number of the last row read from this file. Fields with nested line-end characters will not affect this count.
1703 1704 1705 |
# File 'lib/csv.rb', line 1703 def line @line end |
#lineno ⇒ Object (readonly)
The line number of the last row read from this file. Fields with nested line-end characters will not affect this count.
1703 1704 1705 |
# File 'lib/csv.rb', line 1703 def lineno @lineno end |
#quote_char ⇒ Object (readonly)
The encoded :quote_char
used in parsing and writing. See CSV::new for details.
1634 1635 1636 |
# File 'lib/csv.rb', line 1634 def quote_char @quote_char end |
#row_sep ⇒ Object (readonly)
The encoded :row_sep
used in parsing and writing. See CSV::new for details.
1629 1630 1631 |
# File 'lib/csv.rb', line 1629 def row_sep @row_sep end |
#skip_lines ⇒ Object (readonly)
The regex marking a line as a comment. See CSV::new for details
1639 1640 1641 |
# File 'lib/csv.rb', line 1639 def skip_lines @skip_lines end |
Class Method Details
.filter(input = nil, output = nil, **options) ⇒ Object
:call-seq:
filter( **options ) { |row| ... }
filter( input, **options ) { |row| ... }
filter( input, output, **options ) { |row| ... }
This method is a convenience for building Unix-like filters for CSV data. Each row is yielded to the provided block which can alter it as needed. After the block returns, the row is appended to output
altered or not.
The input
and output
arguments can be anything CSV::new() accepts (generally String or IO objects). If not given, they default to ARGF
and $stdout
.
The options
parameter is also filtered down to CSV::new() after some clever key parsing. Any key beginning with :in_
or :input_
will have that leading identifier stripped and will only be used in the options
Hash for the input
object. Keys starting with :out_
or :output_
affect only output
. All other keys are assigned to both objects.
The :output_row_sep
option
defaults to $INPUT_RECORD_SEPARATOR
($/
).
1150 1151 1152 1153 1154 1155 1156 1157 1158 1159 1160 1161 1162 1163 1164 1165 1166 1167 1168 1169 1170 1171 1172 1173 |
# File 'lib/csv.rb', line 1150 def self.filter(input=nil, output=nil, **) # parse options for input, output, or both , = Hash.new, {row_sep: $INPUT_RECORD_SEPARATOR} .each do |key, value| case key.to_s when /\Ain(?:put)?_(.+)\Z/ [$1.to_sym] = value when /\Aout(?:put)?_(.+)\Z/ [$1.to_sym] = value else [key] = value [key] = value end end # build input and output wrappers input = new(input || ARGF, ) output = new(output || $stdout, ) # read, yield, write input.each do |row| yield row output << row end end |
.foreach(path, **options, &block) ⇒ Object
This method is intended as the primary interface for reading CSV files. You pass a path
and any options
you wish to set for the read. Each row of file will be passed to the provided block
in turn.
The options
parameter can be anything CSV::new() understands. This method also understands an additional :encoding
parameter that you can use to specify the Encoding of the data in the file to be read. You must provide this unless your data is in Encoding::default_external(). CSV will use this to determine how to parse the data. You may provide a second Encoding to have the data transcoded as it is read. For example, encoding: "UTF-32BE:UTF-8"
would read UTF-32BE data from the file but transcode it to UTF-8 before CSV parses it.
1189 1190 1191 1192 1193 1194 |
# File 'lib/csv.rb', line 1189 def self.foreach(path, **, &block) return to_enum(__method__, path, ) unless block_given? open(path, ) do |csv| csv.each(&block) end end |
.generate(str = nil, **options) {|csv| ... } ⇒ Object
:call-seq:
generate( str, **options ) { |csv| ... }
generate( **options ) { |csv| ... }
This method wraps a String you provide, or an empty default String, in a CSV object which is passed to the provided block. You can use the block to append CSV rows to the String and when the block exits, the final String will be returned.
Note that a passed String is modified by this method. Call dup() before passing if you need a new String.
The options
parameter can be anything CSV::new() understands. This method understands an additional :encoding
parameter when not passed a String to set the base Encoding for the output. CSV needs this hint if you plan to output non-ASCII compatible data.
1214 1215 1216 1217 1218 1219 1220 1221 1222 1223 1224 1225 1226 1227 |
# File 'lib/csv.rb', line 1214 def self.generate(str=nil, **) # add a default empty String, if none was given if str str = StringIO.new(str) str.seek(0, IO::SEEK_END) else encoding = [:encoding] str = String.new str.force_encoding(encoding) if encoding end csv = new(str, ) # wrap yield csv # yield for appending csv.string # return final String end |
.generate_line(row, **options) ⇒ Object
This method is a shortcut for converting a single row (Array) into a CSV String.
The options
parameter can be anything CSV::new() understands. This method understands an additional :encoding
parameter to set the base Encoding for the output. This method will try to guess your Encoding from the first non-nil
field in row
, if possible, but you may need to use this parameter as a backup plan.
The :row_sep
option
defaults to $INPUT_RECORD_SEPARATOR
($/
) when calling this method.
1242 1243 1244 1245 1246 1247 1248 1249 1250 1251 |
# File 'lib/csv.rb', line 1242 def self.generate_line(row, **) = {row_sep: $INPUT_RECORD_SEPARATOR}.merge() str = String.new if [:encoding] str.force_encoding([:encoding]) elsif field = row.find { |f| not f.nil? } str.force_encoding(String(field).encoding) end (new(str, ) << row).string end |
.instance(data = $stdout, **options) ⇒ Object
This method will return a CSV instance, just like CSV::new(), but the instance will be cached and returned for all future calls to this method for the same data
object (tested by Object#object_id()) with the same options
.
If a block is given, the instance is passed to the block and the return value becomes the return value of the block.
1110 1111 1112 1113 1114 1115 1116 1117 1118 1119 1120 1121 1122 1123 1124 |
# File 'lib/csv.rb', line 1110 def self.instance(data = $stdout, **) # create a _signature_ for this method call, data object and options sig = [data.object_id] + .values_at(*DEFAULT_OPTIONS.keys.sort_by { |sym| sym.to_s }) # fetch or create the instance for this signature @@instances ||= Hash.new instance = (@@instances[sig] ||= new(data, )) if block_given? yield instance # run block, if given, returning result else instance # or return the instance end end |
.open(filename, mode = "r", **options) ⇒ Object
:call-seq:
open( filename, mode = "rb", **options ) { |faster_csv| ... }
open( filename, **options ) { |faster_csv| ... }
open( filename, mode = "rb", **options )
open( filename, **options )
This method opens an IO object, and wraps that with CSV. This is intended as the primary interface for writing a CSV file.
You must pass a filename
and may optionally add a mode
for Ruby’s open(). You may also pass an optional Hash containing any options
CSV::new() understands as the final argument.
This method works like Ruby’s open() call, in that it will pass a CSV object to a provided block and close it when the block terminates, or it will return the CSV object when no block is provided. (Note: This is different from the Ruby 1.8 CSV library which passed rows to the block. Use CSV::foreach() for that behavior.)
You must provide a mode
with an embedded Encoding designator unless your data is in Encoding::default_external(). CSV will check the Encoding of the underlying IO object (set by the mode
you pass) to determine how to parse the data. You may provide a second Encoding to have the data transcoded as it is read just as you can with a normal call to IO::open(). For example, "rb:UTF-32BE:UTF-8"
would read UTF-32BE data from the file but transcode it to UTF-8 before CSV parses it.
An opened CSV object will delegate to many IO methods for convenience. You may call:
-
binmode()
-
binmode?()
-
close()
-
close_read()
-
close_write()
-
closed?()
-
eof()
-
eof?()
-
external_encoding()
-
fcntl()
-
fileno()
-
flock()
-
flush()
-
fsync()
-
internal_encoding()
-
ioctl()
-
isatty()
-
path()
-
pid()
-
pos()
-
pos=()
-
reopen()
-
seek()
-
stat()
-
sync()
-
sync=()
-
tell()
-
to_i()
-
to_io()
-
truncate()
-
tty?()
1316 1317 1318 1319 1320 1321 1322 1323 1324 1325 1326 1327 1328 1329 1330 1331 1332 1333 1334 1335 1336 1337 1338 1339 1340 1341 1342 1343 1344 1345 1346 |
# File 'lib/csv.rb', line 1316 def self.open(filename, mode="r", **) # wrap a File opened with the remaining +args+ with no newline # decorator file_opts = {universal_newline: false}.merge() begin f = File.open(filename, mode, file_opts) rescue ArgumentError => e raise unless /needs binmode/ =~ e. and mode == "r" mode = "rb" file_opts = {encoding: Encoding.default_external}.merge(file_opts) retry end begin csv = new(f, ) rescue Exception f.close raise end # handle blocks like Ruby's open(), not like the CSV library if block_given? begin yield csv ensure csv.close end else csv end end |
.parse(*args, &block) ⇒ Object
:call-seq:
parse( str, **options ) { |row| ... }
parse( str, **options )
This method can be used to easily parse CSV out of a String. You may either provide a block
which will be called with each row of the String in turn, or just use the returned Array of Arrays (when no block
is given).
You pass your str
to read from, and an optional options
containing anything CSV::new() understands.
1360 1361 1362 1363 1364 1365 1366 1367 1368 1369 1370 1371 |
# File 'lib/csv.rb', line 1360 def self.parse(*args, &block) csv = new(*args) return csv.each(&block) if block_given? # slurp contents, if no block is given begin csv.read ensure csv.close end end |
.parse_line(line, **options) ⇒ Object
This method is a shortcut for converting a single line of a CSV String into an Array. Note that if line
contains multiple rows, anything beyond the first row is ignored.
The options
parameter can be anything CSV::new() understands.
1380 1381 1382 |
# File 'lib/csv.rb', line 1380 def self.parse_line(line, **) new(line, ).shift end |
.read(path, *options) ⇒ Object
Use to slurp a CSV file into an Array of Arrays. Pass the path
to the file and any options
CSV::new() understands. This method also understands an additional :encoding
parameter that you can use to specify the Encoding of the data in the file to be read. You must provide this unless your data is in Encoding::default_external(). CSV will use this to determine how to parse the data. You may provide a second Encoding to have the data transcoded as it is read. For example, encoding: "UTF-32BE:UTF-8"
would read UTF-32BE data from the file but transcode it to UTF-8 before CSV parses it.
1395 1396 1397 |
# File 'lib/csv.rb', line 1395 def self.read(path, *) open(path, *) { |csv| csv.read } end |
.readlines(*args) ⇒ Object
Alias for CSV::read().
1400 1401 1402 |
# File 'lib/csv.rb', line 1400 def self.readlines(*args) read(*args) end |
.table(path, **options) ⇒ Object
1411 1412 1413 1414 1415 |
# File 'lib/csv.rb', line 1411 def self.table(path, **) read( path, { headers: true, converters: :numeric, header_converters: :symbol }.merge() ) end |
Instance Method Details
#<<(row) ⇒ Object Also known as: add_row, puts
The primary write method for wrapped Strings and IOs, row
(an Array or CSV::Row) is converted to CSV and appended to the data source. When a CSV::Row is passed, only the row’s fields() are appended to the output.
The data source must be open for writing.
1732 1733 1734 1735 1736 1737 1738 1739 1740 1741 1742 1743 1744 1745 1746 1747 1748 1749 1750 1751 1752 1753 1754 1755 1756 1757 1758 1759 1760 1761 |
# File 'lib/csv.rb', line 1732 def <<(row) # make sure headers have been assigned if header_row? and [Array, String].include? @use_headers.class and !@write_headers parse_headers # won't read data for Array or String end # handle CSV::Row objects and Hashes row = case row when self.class::Row then row.fields when Hash then @headers.map { |header| row[header] } else row end @headers = row if header_row? @lineno += 1 output = row.map(&@quote).join(@col_sep) + @row_sep # quote and separate if @io.is_a?(StringIO) and output.encoding != (encoding = raw_encoding) if @force_encoding output = output.encode(encoding) elsif (compatible_encoding = Encoding.compatible?(@io.string, output)) @io.set_encoding(compatible_encoding) @io.seek(0, IO::SEEK_END) end end @io << output self # for chaining end |
#convert(name = nil, &converter) ⇒ Object
:call-seq:
convert( name )
convert { |field| ... }
convert { |field, field_info| ... }
You can use this method to install a CSV::Converters built-in, or provide a block that handles a custom conversion.
If you provide a block that takes one argument, it will be passed the field and is expected to return the converted value or the field itself. If your block takes two arguments, it will also be passed a CSV::FieldInfo Struct, containing details about the field. Again, the block should return a converted field or the field itself.
1780 1781 1782 |
# File 'lib/csv.rb', line 1780 def convert(name = nil, &converter) add_converter(:@converters, self.class::Converters, name, &converter) end |
#converters ⇒ Object
Returns the current list of converters in effect. See CSV::new for details. Built-in converters will be returned by name, while others will be returned as is.
1646 1647 1648 1649 1650 1651 |
# File 'lib/csv.rb', line 1646 def converters @converters.map do |converter| name = Converters.rassoc(converter) name ? name.first : converter end end |
#each ⇒ Object
Yields each row of the data source in turn.
Support for Enumerable.
The data source must be open for reading.
1811 1812 1813 1814 1815 1816 1817 1818 1819 |
# File 'lib/csv.rb', line 1811 def each if block_given? while row = shift yield row end else to_enum end end |
#force_quotes? ⇒ Boolean
Returns true
if all output fields are quoted. See CSV::new for details.
1689 |
# File 'lib/csv.rb', line 1689 def force_quotes?() @force_quotes end |
#header_convert(name = nil, &converter) ⇒ Object
:call-seq:
header_convert( name )
header_convert { |field| ... }
header_convert { |field, field_info| ... }
Identical to CSV#convert(), but for header rows.
Note that this method must be called before header rows are read to have any effect.
1795 1796 1797 1798 1799 1800 |
# File 'lib/csv.rb', line 1795 def header_convert(name = nil, &converter) add_converter( :@header_converters, self.class::HeaderConverters, name, &converter ) end |
#header_converters ⇒ Object
Returns the current list of converters in effect for headers. See CSV::new for details. Built-in converters will be returned by name, while others will be returned as is.
1677 1678 1679 1680 1681 1682 |
# File 'lib/csv.rb', line 1677 def header_converters @header_converters.map do |converter| name = HeaderConverters.rassoc(converter) name ? name.first : converter end end |
#header_row? ⇒ Boolean
Returns true
if the next row read will be a header row.
1837 1838 1839 |
# File 'lib/csv.rb', line 1837 def header_row? @use_headers and @headers.nil? end |
#headers ⇒ Object
Returns nil
if headers will not be used, true
if they will but have not yet been read, or the actual headers after they have been read. See CSV::new for details.
1662 1663 1664 |
# File 'lib/csv.rb', line 1662 def headers @headers || true if @use_headers end |
#inspect ⇒ Object
Returns a simplified description of the key CSV attributes in an ASCII compatible String.
2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 2026 2027 2028 2029 2030 2031 2032 2033 2034 2035 2036 2037 2038 2039 2040 2041 2042 2043 2044 |
# File 'lib/csv.rb', line 2011 def inspect str = ["<#", self.class.to_s, " io_type:"] # show type of wrapped IO if @io == $stdout then str << "$stdout" elsif @io == $stdin then str << "$stdin" elsif @io == $stderr then str << "$stderr" else str << @io.class.to_s end # show IO.path(), if available if @io.respond_to?(:path) and (p = @io.path) str << " io_path:" << p.inspect end # show encoding str << " encoding:" << @encoding.name # show other attributes %w[ lineno col_sep row_sep quote_char skip_blanks liberal_parsing ].each do |attr_name| if a = instance_variable_get("@#{attr_name}") str << " " << attr_name << ":" << a.inspect end end if @use_headers str << " headers:" << headers.inspect end str << ">" begin str.join('') rescue # any encoding error str.map do |s| e = Encoding::Converter.asciicompat_encoding(s.encoding) e ? s.encode(e) : s.force_encoding("ASCII-8BIT") end.join('') end end |
#liberal_parsing? ⇒ Boolean
Returns true
if illegal input is handled. See CSV::new for details.
1691 |
# File 'lib/csv.rb', line 1691 def liberal_parsing?() @liberal_parsing end |
#read ⇒ Object Also known as: readlines
Slurps the remaining rows and returns an Array of Arrays.
The data source must be open for reading.
1826 1827 1828 1829 1830 1831 1832 1833 |
# File 'lib/csv.rb', line 1826 def read rows = to_a if @use_headers Table.new(rows) else rows end end |
#return_headers? ⇒ Boolean
Returns true
if headers will be returned as a row of results. See CSV::new for details.
1669 |
# File 'lib/csv.rb', line 1669 def return_headers?() @return_headers end |
#rewind ⇒ Object
Rewinds the underlying IO object and resets CSV’s lineno() counter.
1716 1717 1718 1719 1720 1721 |
# File 'lib/csv.rb', line 1716 def rewind @headers = nil @lineno = 0 @io.rewind end |
#shift ⇒ Object Also known as: gets, readline
The primary read method for wrapped Strings and IOs, a single row is pulled from the data source, parsed and returned as an Array of fields (if header rows are not used) or a CSV::Row (when header rows are used).
The data source must be open for reading.
1848 1849 1850 1851 1852 1853 1854 1855 1856 1857 1858 1859 1860 1861 1862 1863 1864 1865 1866 1867 1868 1869 1870 1871 1872 1873 1874 1875 1876 1877 1878 1879 1880 1881 1882 1883 1884 1885 1886 1887 1888 1889 1890 1891 1892 1893 1894 1895 1896 1897 1898 1899 1900 1901 1902 1903 1904 1905 1906 1907 1908 1909 1910 1911 1912 1913 1914 1915 1916 1917 1918 1919 1920 1921 1922 1923 1924 1925 1926 1927 1928 1929 1930 1931 1932 1933 1934 1935 1936 1937 1938 1939 1940 1941 1942 1943 1944 1945 1946 1947 1948 1949 1950 1951 1952 1953 1954 1955 1956 1957 1958 1959 1960 1961 1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975 1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 |
# File 'lib/csv.rb', line 1848 def shift ######################################################################### ### This method is purposefully kept a bit long as simple conditional ### ### checks are faster than numerous (expensive) method calls. ### ######################################################################### # handle headers not based on document content if header_row? and @return_headers and [Array, String].include? @use_headers.class if @unconverted_fields return add_unconverted_fields(parse_headers, Array.new) else return parse_headers end end # # it can take multiple calls to <tt>@io.gets()</tt> to get a full line, # because of \r and/or \n characters embedded in quoted fields # in_extended_col = false csv = Array.new loop do # add another read to the line unless parse = @io.gets(@row_sep) return nil end if in_extended_col @line.concat(parse) else @line = parse.clone end parse.sub!(@parsers[:line_end], "") if csv.empty? # # I believe a blank line should be an <tt>Array.new</tt>, not Ruby 1.8 # CSV's <tt>[nil]</tt> # if parse.empty? @lineno += 1 if @skip_blanks next elsif @unconverted_fields return add_unconverted_fields(Array.new, Array.new) elsif @use_headers return self.class::Row.new(Array.new, Array.new) else return Array.new end end end next if @skip_lines and @skip_lines.match parse parts = parse.split(@col_sep, -1) if parts.empty? if in_extended_col csv[-1] << @col_sep # will be replaced with a @row_sep after the parts.each loop else csv << nil end end # This loop is the hot path of csv parsing. Some things may be non-dry # for a reason. Make sure to benchmark when refactoring. parts.each do |part| if in_extended_col # If we are continuing a previous column if part.end_with?(@quote_char) && part.count(@quote_char) % 2 != 0 # extended column ends csv.last << part[0..-2] if csv.last =~ @parsers[:stray_quote] raise MalformedCSVError, "Missing or stray quote in line #{lineno + 1}" end csv.last.gsub!(@double_quote_char, @quote_char) in_extended_col = false else csv.last << part << @col_sep end elsif part.start_with?(@quote_char) # If we are starting a new quoted column if part.count(@quote_char) % 2 != 0 # start an extended column csv << (part[1..-1] << @col_sep) in_extended_col = true elsif part.end_with?(@quote_char) # regular quoted column csv << part[1..-2] if csv.last =~ @parsers[:stray_quote] raise MalformedCSVError, "Missing or stray quote in line #{lineno + 1}" end csv.last.gsub!(@double_quote_char, @quote_char) elsif @liberal_parsing csv << part else raise MalformedCSVError, "Missing or stray quote in line #{lineno + 1}" end elsif part =~ @parsers[:quote_or_nl] # Unquoted field with bad characters. if part =~ @parsers[:nl_or_lf] raise MalformedCSVError, "Unquoted fields do not allow " + "\\r or \\n (line #{lineno + 1})." else if @liberal_parsing csv << part else raise MalformedCSVError, "Illegal quoting in line #{lineno + 1}." end end else # Regular ole unquoted field. csv << (part.empty? ? nil : part) end end # Replace tacked on @col_sep with @row_sep if we are still in an extended # column. csv[-1][-1] = @row_sep if in_extended_col if in_extended_col # if we're at eof?(), a quoted field wasn't closed... if @io.eof? raise MalformedCSVError, "Unclosed quoted field on line #{lineno + 1}." elsif @field_size_limit and csv.last.size >= @field_size_limit raise MalformedCSVError, "Field size exceeded on line #{lineno + 1}." end # otherwise, we need to loop and pull some more data to complete the row else @lineno += 1 # save fields unconverted fields, if needed... unconverted = csv.dup if @unconverted_fields # convert fields, if needed... csv = convert_fields(csv) unless @use_headers or @converters.empty? # parse out header rows and handle CSV::Row conversions... csv = parse_headers(csv) if @use_headers # inject unconverted fields and accessor, if requested... if @unconverted_fields and not csv.respond_to? :unconverted_fields add_unconverted_fields(csv, unconverted) end # return the results break csv end end end |
#skip_blanks? ⇒ Boolean
Returns true
blank lines are skipped by the parser. See CSV::new for details.
1687 |
# File 'lib/csv.rb', line 1687 def skip_blanks?() @skip_blanks end |
#unconverted_fields? ⇒ Boolean
Returns true
if unconverted_fields() to parsed results. See CSV::new for details.
1656 |
# File 'lib/csv.rb', line 1656 def unconverted_fields?() @unconverted_fields end |
#write_headers? ⇒ Boolean
Returns true
if headers are written in output. See CSV::new for details.
1671 |
# File 'lib/csv.rb', line 1671 def write_headers?() @write_headers end |