Class: CSV
Relationships & Source Files | |
Namespace Children | |
Modules:
| |
Classes:
| |
Exceptions:
| |
Super Chains via Extension / Inclusion / Inheritance | |
Class Chain:
self,
Forwardable
|
|
Instance Chain:
self,
Enumerable
|
|
Inherits: | Object |
Defined in: | lib/csv.rb, lib/csv/delete_suffix.rb, lib/csv/fields_converter.rb, lib/csv/match_p.rb, lib/csv/parser.rb, lib/csv/row.rb, lib/csv/table.rb, lib/csv/version.rb, lib/csv/writer.rb |
Overview
This class provides a complete interface to CSV
files and data. It offers tools to enable you to read and write to and from Strings or IO objects, as needed.
The most generic interface of the library is:
csv = CSV.new(string_or_io, ** )
# Reading: IO object should be open for read
csv.read # => array of rows
# or
csv.each do |row|
# ...
end
# or
row = csv.shift
# Writing: IO object should be open for write
csv << row
There are several specialized class methods for one-statement reading or writing, described in the Specialized Methods section.
If a ::String
is passed into .new, it is internally wrapped into a StringIO object.
options
can be used for specifying the particular CSV
flavor (column separators, row separators, value quoting and so on), and for data conversion, see Data Conversion section for the description of the latter.
Specialized Methods
Reading
# From a file: all at once
arr_of_rows = CSV.read("path/to/file.csv", ** )
# iterator-style:
CSV.foreach("path/to/file.csv", ** ) do |row|
# ...
end
# From a string
arr_of_rows = CSV.parse("CSV,data,String", ** )
# or
CSV.parse("CSV,data,String", ** ) do |row|
# ...
end
Writing
# To a file
CSV.open("path/to/file.csv", "wb") do |csv|
csv << ["row", "of", "CSV", "data"]
csv << ["another", "row"]
# ...
end
# To a String
csv_string = CSV.generate do |csv|
csv << ["row", "of", "CSV", "data"]
csv << ["another", "row"]
# ...
end
Shortcuts
# Core extensions for converting one line
csv_string = ["CSV", "data"].to_csv # to CSV
csv_array = "CSV,String".parse_csv # from CSV
# CSV() method
CSV { |csv_out| csv_out << %w{my data here} } # to $stdout
CSV(csv = "") { |csv_str| csv_str << %w{my data here} } # to a String
CSV($stderr) { |csv_err| csv_err << %w{my data here} } # to $stderr
CSV($stdin) { |csv_in| csv_in.each { |row| p row } } # from $stdin
Data Conversion
CSV with headers
CSV
allows to specify column names of CSV
file, whether they are in data, or provided separately. If headers are specified, reading methods return an instance of Table
, consisting of Row
.
# Headers are part of data
data = CSV.parse(<<~ROWS, headers: true)
Name,Department,Salary
Bob,Engineering,1000
Jane,Sales,2000
John,Management,5000
ROWS
data.class #=> CSV::Table
data.first #=> #<CSV::Row "Name":"Bob" "Department":"Engineering" "Salary":"1000">
data.first.to_h #=> {"Name"=>"Bob", "Department"=>"Engineering", "Salary"=>"1000"}
# Headers provided by developer
data = CSV.parse('Bob,Engineering,1000', headers: %i[name department salary])
data.first #=> #<CSV::Row name:"Bob" department:"Engineering" salary:"1000">
Typed data reading
CSV
allows to provide a set of data converters e.g. transformations to try on input data. Converter could be a symbol from Converters constant’s keys, or lambda.
# Without any converters:
CSV.parse('Bob,2018-03-01,100')
#=> [["Bob", "2018-03-01", "100"]]
# With built-in converters:
CSV.parse('Bob,2018-03-01,100', converters: %i[numeric date])
#=> [["Bob", #<Date: 2018-03-01>, 100]]
# With custom converters:
CSV.parse('Bob,2018-03-01,100', converters: [->(v) { Time.parse(v) rescue v }])
#=> [["Bob", 2018-03-01 00:00:00 +0200, "100"]]
CSV and Character Encodings (M17n or Multilingualization)
This new CSV
parser is m17n savvy. The parser works in the Encoding of the IO or ::String
object being read from or written to. Your data is never transcoded (unless you ask Ruby to transcode it for you) and will literally be parsed in the Encoding it is in. Thus CSV will return Arrays or Rows of Strings in the Encoding of your data. This is accomplished by transcoding the parser itself into your Encoding.
Some transcoding must take place, of course, to accomplish this multiencoding support. For example, :col_sep
, :row_sep
, and :quote_char
must be transcoded to match your data. Hopefully this makes the entire process feel transparent, since CSV’s defaults should just magically work for your data. However, you can set these values manually in the target Encoding to avoid the translation.
It’s also important to note that while all of CSV’s core parser is now Encoding agnostic, some features are not. For example, the built-in converters will try to transcode data to UTF-8 before making conversions. Again, you can provide custom converters that are aware of your Encodings to avoid this translation. It’s just too hard for me to support native conversions in all of Ruby’s Encodings.
Anyway, the practical side of this is simple: make sure IO and ::String
objects passed into CSV
have the proper Encoding set and everything should just work. CSV
methods that allow you to open IO objects (CSV::foreach(), .open(), .read(), and .readlines()) do allow you to specify the Encoding.
One minor exception comes when generating CSV
into a ::String
with an Encoding that is not ASCII compatible. There’s no existing data for CSV
to use to prepare itself and thus you will probably need to manually specify the desired Encoding for most of those cases. It will try to guess using the fields in a row of output though, when using .generate_line() or Array#to_csv().
I try to point out any other Encoding issues in the documentation of methods as they come up.
This has been tested to the best of my ability with all non-“dummy” Encodings Ruby ships with. However, it is brave new code and may have some bugs. Please feel free to report any issues you find with it.
Constant Summary
-
ConverterEncoding =
The encoding used by all converters.
Encoding.find("UTF-8")
-
Converters =
This Hash holds the built-in converters of
CSV
that can be accessed by name. You can selectConverters
with #convert() or through theoptions
Hash passed to .new().:integer
-
Converts any field Integer() accepts.
:float
-
Converts any field Float() accepts.
:numeric
-
A combination of
:integer
and:float
. :date
-
Converts any field
Date.parse
() accepts. :date_time
-
Converts any field
DateTime.parse
() accepts. :all
-
All built-in converters. A combination of
:date_time
and:numeric
.
All built-in converters transcode field data to UTF-8 before attempting a conversion. If your data cannot be transcoded to UTF-8 the conversion will fail and the field will remain unchanged.
This Hash is intentionally left unfrozen and users should feel free to add values to it that can be accessed by all
CSV
objects.To add a combo field, the value should be an
::Array
of names. Combo fields can be nested with other combo fields.{ integer: lambda { |f| Integer(f.encode(ConverterEncoding)) rescue f }, float: lambda { |f| Float(f.encode(ConverterEncoding)) rescue f }, numeric: [:integer, :float], date: lambda { |f| begin e = f.encode(ConverterEncoding) e.match?(DateMatcher) ? Date.parse(e) : f rescue # encoding conversion or date parse errors f end }, date_time: lambda { |f| begin e = f.encode(ConverterEncoding) e.match?(DateTimeMatcher) ? DateTime.parse(e) : f rescue # encoding conversion or date parse errors f end }, all: [:date_time, :numeric], }
-
DEFAULT_OPTIONS =
The options used when no overrides are given by calling code. They are:
:col_sep
-
","
:row_sep
-
:auto
:quote_char
-
'"'
:field_size_limit
-
nil
:converters
-
nil
:unconverted_fields
-
nil
:headers
-
false
:return_headers
-
false
:header_converters
-
nil
:skip_blanks
-
false
:force_quotes
-
false
:skip_lines
-
nil
:liberal_parsing
-
false
:quote_empty
-
true
{ col_sep: ",", row_sep: :auto, quote_char: '"', field_size_limit: nil, converters: nil, unconverted_fields: nil, headers: false, return_headers: false, header_converters: nil, skip_blanks: false, force_quotes: false, skip_lines: nil, liberal_parsing: false, quote_empty: true, }.freeze
-
DateMatcher =
A Regexp used to find and convert some common Date formats.
/ \A(?: (\w,?\s)?\w\s\d{1,2},?\s+\d{2,4} | \d{4}-\d{2}-\d{2} )\z /x
-
DateTimeMatcher =
A Regexp used to find and convert some common DateTime formats.
/ \A(?: (\w,?\s)?\w\s\d{1,2}\s\d{1,2}:\d{1,2}:\d{1,2},?\s\d{2,4} | \d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2} | # ISO-8601 \d{4}-\d{2}-\d{2} (?:T\d{2}:\d{2}(?::\d{2}(?:\.\d)?(?:[-]\d{2}(?::\d{2})|Z)?)?)? )\z /x
-
HeaderConverters =
This Hash holds the built-in header converters of
CSV
that can be accessed by name. You can selectHeaderConverters
with #header_convert() or through theoptions
Hash passed to .new().:downcase
-
Calls downcase() on the header
::String
. :symbol
-
Leading/trailing spaces are dropped, string is downcased, remaining spaces are replaced with underscores, non-word characters are dropped, and finally to_sym() is called.
All built-in header converters transcode header data to UTF-8 before attempting a conversion. If your data cannot be transcoded to UTF-8 the conversion will fail and the header will remain unchanged.
This Hash is intentionally left unfrozen and users should feel free to add values to it that can be accessed by all
CSV
objects.To add a combo field, the value should be an
::Array
of names. Combo fields can be nested with other combo fields.{ downcase: lambda { |h| h.encode(ConverterEncoding).downcase }, symbol: lambda { |h| h.encode(ConverterEncoding).downcase.gsub(/[^\s\w]+/, "").strip. gsub(/\s+/, "_").to_sym } }
-
VERSION =
The version of the installed library.
"3.1.2"
Class Method Summary
-
.filter(**options) {|row| ... }
This method is a convenience for building Unix-like filters for
CSV
data. -
.foreach(path, mode = "r", **options, &block)
This method is intended as the primary interface for reading
CSV
files. - .generate(str, **options) {|csv| ... }
-
.generate_line(row, **options)
This method is a shortcut for converting a single row (Array) into a
CSV
String. -
.instance(data = $stdout, **options)
This method will return a
CSV
instance, just like .new(), but the instance will be cached and returned for all future calls to this method for the samedata
object (tested byObject#object_id
()) with the sameoptions
. -
.new(data, col_sep: ",", row_sep: :auto, quote_char: '"', field_size_limit: nil, converters: nil, unconverted_fields: nil, headers: false, return_headers: false, write_headers: nil, header_converters: nil, skip_blanks: false, force_quotes: false, skip_lines: nil, liberal_parsing: false, internal_encoding: nil, external_encoding: nil, encoding: nil, nil_value: nil, empty_value: "", quote_empty: true, write_converters: nil, write_nil_value: nil, write_empty_value: "", strip: false) ⇒ CSV
constructor
This constructor will wrap either a
::String
or IO object passed indata
for reading and/or writing. -
.open(filename, mode = "rb", **options) {|faster_csv| ... }
This method opens an IO object, and wraps that with
CSV
. -
.parse(str, **options) {|row| ... }
This method can be used to easily parse
CSV
out of a::String
. -
.parse_line(line, **options)
This method is a shortcut for converting a single line of a
CSV
String into an::Array
. -
.read(path, **options)
Use to slurp a
CSV
file into an::Array
of Arrays. -
.readlines(path, **options)
Alias for .read().
-
.table(path, **options)
A shortcut for:
Instance Attribute Summary
- #binmode? ⇒ Boolean readonly
-
#encoding
readonly
The Encoding
CSV
is parsing or writing in. -
#eof
readonly
Alias for #eof?.
-
#force_quotes? ⇒ Boolean
readonly
Returns
true
if all output fields are quoted. -
#header_row? ⇒ Boolean
readonly
Returns
true
if the next row read will be a header row. -
#liberal_parsing? ⇒ Boolean
readonly
Returns
true
if illegal input is handled. -
#return_headers? ⇒ Boolean
readonly
Returns
true
if headers will be returned as a row of results. -
#skip_blanks? ⇒ Boolean
readonly
Returns
true
blank lines are skipped by the parser. -
#unconverted_fields? ⇒ Boolean
readonly
Returns
true
if unconverted_fields() to parsed results. -
#write_headers? ⇒ Boolean
readonly
Returns
true
if headers are written in output.
Instance Method Summary
- #<<(row) (also: #add_row, #puts)
-
#add_row(row)
Alias for #<<.
-
#col_sep
The encoded
:col_sep
used in parsing and writing. -
#convert(name)
You can use this method to install a Converters built-in, or provide a block that handles a custom conversion.
-
#converters
Returns the current list of converters in effect.
-
#each(&block)
Yields each row of the data source in turn.
- #eof? ⇒ Boolean (also: #eof) readonly
-
#field_size_limit
The limit for field size, if any.
- #flock(*args)
-
#gets
Alias for #shift.
-
#header_convert(name)
Identical to #convert(), but for header rows.
-
#header_converters
Returns the current list of converters in effect for headers.
-
#headers
Returns
nil
if headers will not be used,true
if they will but have not yet been read, or the actual headers after they have been read. -
#inspect
Returns a simplified description of the key
CSV
attributes in an ASCII compatible::String
. - #ioctl(*args)
-
#line
The last row read from this file.
-
#lineno
The line number of the last row read from this file.
- #path
-
#puts(row)
Alias for #<<.
-
#quote_char
The encoded
:quote_char
used in parsing and writing. -
#read
(also: #readlines)
Slurps the remaining rows and returns an
::Array
of Arrays. -
#readline
Alias for #shift.
-
#readlines
Alias for #read.
-
#rewind
Rewinds the underlying IO object and resets CSV’s lineno() counter.
-
#row_sep
The encoded
:row_sep
used in parsing and writing. - #shift (also: #gets, #readline)
-
#skip_lines
The regex marking a line as a comment.
- #stat(*args)
- #to_i
- #to_io
- #build_fields_converter(initial_converters, options) private
- #build_header_fields_converter private
- #build_parser_fields_converter private
- #build_writer_fields_converter private
-
#convert_fields(fields, headers = false)
private
Processes
fields
with@converters
, or@header_converters
if #headers is passed astrue
, returning the converted field set. - #determine_encoding(encoding, internal_encoding) private
- #header_fields_converter private
- #normalize_converters(converters) private
- #parser private
- #parser_enumerator private
- #parser_fields_converter private
- #parser_options private
-
#raw_encoding
private
Returns the encoding of the internal IO object.
- #writer private
- #writer_fields_converter private
- #writer_options private
Constructor Details
.new(data, col_sep: ",", row_sep: :auto, quote_char: '"', field_size_limit: nil, converters: nil, unconverted_fields: nil, headers: false, return_headers: false, write_headers: nil, header_converters: nil, skip_blanks: false, force_quotes: false, skip_lines: nil, liberal_parsing: false, internal_encoding: nil, external_encoding: nil, encoding: nil, nil_value: nil, empty_value: "", quote_empty: true, write_converters: nil, write_nil_value: nil, write_empty_value: "", strip: false) ⇒ CSV
This constructor will wrap either a ::String
or IO object passed in data
for reading and/or writing. In addition to the CSV
instance methods, several IO methods are delegated. (See .open() for a complete list.) If you pass a ::String
for data
, you can later retrieve it (after writing to it, for example) with CSV.string
().
Note that a wrapped ::String
will be positioned at the beginning (for reading). If you want it at the end (for writing), use .generate(). If you want any other positioning, pass a preset StringIO object instead.
You may set any reading and/or writing preferences in the options
Hash. Available options are:
:col_sep
-
The String placed between each field. This String will be transcoded into the data’s Encoding before parsing.
:row_sep
-
The String appended to the end of each row. This can be set to the special
:auto
setting, which requests that CSV automatically discover this from the data. Auto-discovery reads ahead in the data looking for the next"\r\n"
,"\n"
, or"\r"
sequence. A sequence will be selected even if it occurs in a quoted field, assuming that you would have the same line endings there. If none of those sequences is found,data
isARGF
,STDIN
,STDOUT
, orSTDERR
, or the stream is only available for output, the default$INPUT_RECORD_SEPARATOR
($/
) is used. Obviously, discovery takes a little time. Set manually if speed is important. Also note that IO objects should be opened in binary mode on Windows if this feature will be used as the line-ending translation can cause problems with resetting the document position to where it was before the read ahead. This String will be transcoded into the data’s Encoding before parsing. :quote_char
-
The character used to quote fields. This has to be a single character String. This is useful for application that incorrectly use
'
as the quote character instead of the correct"
. CSV will always consider a double sequence of this character to be an escaped quote. This String will be transcoded into the data’s Encoding before parsing. :field_size_limit
-
This is a maximum size
CSV
will read ahead looking for the closing quote for a field. (In truth, it reads to the first line ending beyond this size.) If a quote cannot be found within the limit CSV will raise a MalformedCSVError, assuming the data is faulty. You can use this limit to prevent what are effectively DoS attacks on the parser. However, this limit can cause a legitimate parse to fail and thus is set tonil
, or off, by default. :converters
-
An Array of names from the Converters Hash and/or lambdas that handle custom conversion. A single converter doesn’t have to be in an Array. All built-in converters try to transcode fields to UTF-8 before converting. The conversion will fail if the data cannot be transcoded, leaving the field unchanged.
:unconverted_fields
-
If set to
true
, an unconverted_fields() method will be added to all returned rows (Array or CSV::Row) that will return the fields as they were before conversion. Note that:headers
supplied by Array or String were not fields of the document and thus will have an empty Array attached. :headers
-
If set to
:first_row
ortrue
, the initial row of the CSV file will be treated as a row of headers. If set to an Array, the contents will be used as the headers. If set to a String, the String is run through a call of CSV::parse_line() with the same:col_sep
,:row_sep
, and:quote_char
as this instance to produce an Array of headers. This setting causes CSV#shift() to return rows as CSV::Row objects instead of Arrays and CSV#read() to return CSV::Table objects instead of an Array of Arrays. :return_headers
-
When
false
, header rows are silently swallowed. If set totrue
, header rows are returned in a CSV::Row object with identical headers and fields (save that the fields do not go through the converters). :write_headers
-
When
true
and:headers
is set, a header row will be added to the output. :header_converters
-
Identical in functionality to
:converters
save that the conversions are only made to header rows. All built-in converters try to transcode headers to UTF-8 before converting. The conversion will fail if the data cannot be transcoded, leaving the header unchanged. :skip_blanks
-
When setting a
true
value,CSV
will skip over any empty rows. Note that this setting will not skip rows that contain column separators, even if the rows contain no actual data. If you want to skip rows that contain separators but no content, consider using:skip_lines
, or inspecting fields.compact.empty? on each row. :force_quotes
-
When setting a
true
value,CSV
will quote all CSV fields it creates. :skip_lines
-
When setting an object responding to
match
, every line matching it is considered a comment and ignored during parsing. When set to a String, it is first converted to a Regexp. When set tonil
no line is considered a comment. If the passed object does not respond tomatch
,ArgumentError
is thrown. :liberal_parsing
-
When setting a
true
value,CSV
will attempt to parse input not conformant with RFC 4180, such as double quotes in unquoted fields. :nil_value
-
When set an object, any values of an empty field is replaced by the set object, not nil.
:empty_value
-
When setting an object, any values of a blank string field is replaced by the set object.
:quote_empty
-
When setting a
true
value,CSV
will quote empty values with double quotes. Whenfalse
, CSV will emit an empty string for an empty field value. :write_converters
-
Converts values on each line with the specified
Proc
object(s), which receive a::String
value and return a::String
ornil
value. When an array is specified, each converter will be applied in order. :write_nil_value
-
When a
::String
value,nil
value(s) on each line will be replaced with the specified value. :write_empty_value
-
When a
::String
ornil
value, empty value(s) on each line will be replaced with the specified value. :strip
-
When setting a
true
value,CSV
will strip “trnfv” around the values. If you specify a string instead oftrue
, CSV will strip string. The length of the string must be 1.
See DEFAULT_OPTIONS for the default settings.
Options cannot be overridden in the instance methods for performance reasons, so be sure to set what you want here.
# File 'lib/csv.rb', line 921
def initialize(data, col_sep: ",", row_sep: :auto, quote_char: '"', field_size_limit: nil, converters: nil, unconverted_fields: nil, headers: false, return_headers: false, write_headers: nil, header_converters: nil, skip_blanks: false, force_quotes: false, skip_lines: nil, liberal_parsing: false, internal_encoding: nil, external_encoding: nil, encoding: nil, nil_value: nil, empty_value: "", quote_empty: true, write_converters: nil, write_nil_value: nil, write_empty_value: "", strip: false) raise ArgumentError.new("Cannot parse nil as CSV") if data.nil? if data.is_a?(String) @io = StringIO.new(data) @io.set_encoding(encoding || data.encoding) else @io = data end @encoding = determine_encoding(encoding, internal_encoding) @base_fields_converter_options = { nil_value: nil_value, empty_value: empty_value, } @write_fields_converter_options = { nil_value: write_nil_value, empty_value: write_empty_value, } @initial_converters = converters @initial_header_converters = header_converters @initial_write_converters = write_converters @parser_options = { column_separator: col_sep, row_separator: row_sep, quote_character: quote_char, field_size_limit: field_size_limit, unconverted_fields: unconverted_fields, headers: headers, return_headers: return_headers, skip_blanks: skip_blanks, skip_lines: skip_lines, liberal_parsing: liberal_parsing, encoding: @encoding, nil_value: nil_value, empty_value: empty_value, strip: strip, } @parser = nil @parser_enumerator = nil @eof_error = nil @writer_options = { encoding: @encoding, force_encoding: (not encoding.nil?), force_quotes: force_quotes, headers: headers, write_headers: write_headers, column_separator: col_sep, row_separator: row_sep, quote_character: quote_char, quote_empty: quote_empty, } @writer = nil writer if @writer_options[:write_headers] end
Class Method Details
.filter(**options) {|row| ... }
.filter(input, **options) {|row| ... }
.filter(input, output, **options) {|row| ... }
This method is a convenience for building Unix-like filters for CSV
data. Each row is yielded to the provided block which can alter it as needed. After the block returns, the row is appended to output
altered or not.
The input
and output
arguments can be anything .new() accepts (generally ::String
or IO objects). If not given, they default to ARGF
and $stdout
.
The options
parameter is also filtered down to .new() after some clever key parsing. Any key beginning with :in_
or :input_
will have that leading identifier stripped and will only be used in the options
Hash for the input
object. Keys starting with :out_
or :output_
affect only output
. All other keys are assigned to both objects.
The :output_row_sep
option
defaults to $INPUT_RECORD_SEPARATOR
($/
).
# File 'lib/csv.rb', line 469
def filter(input=nil, output=nil, ** ) # parse options for input, output, or both , = Hash.new, {row_sep: $INPUT_RECORD_SEPARATOR} .each do |key, value| case key.to_s when /\Ain(?:put)?_(.+)\Z/ [$1.to_sym] = value when /\Aout(?:put)?_(.+)\Z/ [$1.to_sym] = value else [key] = value [key] = value end end # build input and output wrappers input = new(input || ARGF, ** ) output = new(output || $stdout, ** ) # read, yield, write input.each do |row| yield row output << row end end
.foreach(path, mode = "r", **options, &block)
This method is intended as the primary interface for reading CSV
files. You pass a #path and any options
you wish to set for the read. Each row of file will be passed to the provided block
in turn.
The options
parameter can be anything .new() understands. This method also understands an additional :encoding
parameter that you can use to specify the Encoding of the data in the file to be read. You must provide this unless your data is in Encoding.default_external
(). CSV
will use this to determine how to parse the data. You may provide a second Encoding to have the data transcoded as it is read. For example, encoding: "UTF-32BE:UTF-8"
would read UTF-32BE data from the file but transcode it to UTF-8 before CSV
parses it.
.generate(str, **options) {|csv| ... }
.generate(**options) {|csv| ... }
This method wraps a ::String
you provide, or an empty default ::String
, in a CSV
object which is passed to the provided block. You can use the block to append CSV
rows to the ::String
and when the block exits, the final ::String
will be returned.
Note that a passed ::String
is modified by this method. Call dup() before passing if you need a new ::String
.
The options
parameter can be anything .new() understands. This method understands an additional :encoding
parameter when not passed a ::String
to set the base Encoding for the output. CSV
needs this hint if you plan to output non-ASCII compatible data.
# File 'lib/csv.rb', line 533
def generate(str=nil, ** ) # add a default empty String, if none was given if str str = StringIO.new(str) str.seek(0, IO::SEEK_END) else encoding = [:encoding] str = +"" str.force_encoding(encoding) if encoding end csv = new(str, ** ) # wrap yield csv # yield for appending csv.string # return final String end
.generate_line(row, **options)
This method is a shortcut for converting a single row (Array) into a CSV
::String
.
The options
parameter can be anything .new() understands. This method understands an additional :encoding
parameter to set the base Encoding for the output. This method will try to guess your Encoding from the first non-nil
field in row
, if possible, but you may need to use this parameter as a backup plan.
The :row_sep
option
defaults to $INPUT_RECORD_SEPARATOR
($/
) when calling this method.
.instance(data = $stdout, **options)
This method will return a CSV
instance, just like .new(), but the instance will be cached and returned for all future calls to this method for the same data
object (tested by Object#object_id
()) with the same options
.
If a block is given, the instance is passed to the block and the return value becomes the return value of the block.
# File 'lib/csv.rb', line 429
def instance(data = $stdout, ** ) # create a _signature_ for this method call, data object and options sig = [data.object_id] + .values_at(*DEFAULT_OPTIONS.keys.sort_by { |sym| sym.to_s }) # fetch or create the instance for this signature @@instances ||= Hash.new instance = (@@instances[sig] ||= new(data, ** )) if block_given? yield instance # run block, if given, returning result else instance # or return the instance end end
.open(filename, mode = "rb", **options) {|faster_csv| ... }
.open(filename, **options) {|faster_csv| ... }
.open(filename, mode = "rb", **options)
.open(filename, **options)
This method opens an IO object, and wraps that with CSV
. This is intended as the primary interface for writing a CSV
file.
You must pass a filename
and may optionally add a mode
for Ruby’s open(). You may also pass an optional Hash containing any options
.new() understands as the final argument.
This method works like Ruby’s open() call, in that it will pass a CSV
object to a provided block and close it when the block terminates, or it will return the CSV
object when no block is provided. (Note: This is different from the Ruby 1.8 CSV
library which passed rows to the block. Use .foreach() for that behavior.)
You must provide a mode
with an embedded Encoding designator unless your data is in Encoding.default_external
(). CSV
will check the Encoding of the underlying IO object (set by the mode
you pass) to determine how to parse the data. You may provide a second Encoding to have the data transcoded as it is read just as you can with a normal call to IO.open
(). For example, "rb:UTF-32BE:UTF-8"
would read UTF-32BE data from the file but transcode it to UTF-8 before CSV
parses it.
An opened CSV
object will delegate to many IO methods for convenience. You may call:
-
binmode()
-
binmode?()
-
close()
-
close_read()
-
close_write()
-
closed?()
-
eof()
-
eof?()
-
external_encoding()
-
fcntl()
-
fileno()
-
flock()
-
flush()
-
fsync()
-
internal_encoding()
-
ioctl()
-
isatty()
-
path()
-
pid()
-
pos()
-
pos=()
-
reopen()
-
seek()
-
stat()
-
sync()
-
sync=()
-
tell()
-
to_i()
-
to_io()
-
truncate()
-
tty?()
# File 'lib/csv.rb', line 635
def open(filename, mode="r", ** ) # wrap a File opened with the remaining args with no newline # decorator file_opts = {universal_newline: false}.merge( ) begin f = File.open(filename, mode, **file_opts) rescue ArgumentError => e raise unless /needs binmode/.match?(e. ) and mode == "r" mode = "rb" file_opts = {encoding: Encoding.default_external}.merge(file_opts) retry end begin csv = new(f, ** ) rescue Exception f.close raise end # handle blocks like Ruby's open(), not like the CSV library if block_given? begin yield csv ensure csv.close end else csv end end
.parse(str, **options) {|row| ... }
.parse(str, **options)
This method can be used to easily parse CSV
out of a ::String
. You may either provide a block
which will be called with each row of the ::String
in turn, or just use the returned ::Array
of Arrays (when no block
is given).
You pass your str
to read from, and an optional options
containing anything .new() understands.
.parse_line(line, **options)
.read(path, **options)
Use to slurp a CSV
file into an ::Array
of Arrays. Pass the #path to the file and any options
.new() understands. This method also understands an additional :encoding
parameter that you can use to specify the Encoding of the data in the file to be read. You must provide this unless your data is in Encoding.default_external
(). CSV
will use this to determine how to parse the data. You may provide a second Encoding to have the data transcoded as it is read. For example, encoding: "UTF-32BE:UTF-8"
would read UTF-32BE data from the file but transcode it to UTF-8 before CSV
parses it.
.readlines(path, **options)
Alias for .read().
.table(path, **options)
Instance Attribute Details
#binmode? ⇒ Boolean
(readonly)
[ GitHub ]
# File 'lib/csv.rb', line 1161
def binmode? if @io.respond_to?(:binmode?) @io.binmode? else false end end
#encoding (readonly)
The Encoding CSV
is parsing or writing in. This will be the Encoding you receive parsed data in and/or the Encoding data will be written in.
# File 'lib/csv.rb', line 1130
attr_reader :encoding
#eof (readonly)
Alias for #eof?.
# File 'lib/csv.rb', line 1209
alias_method :eof, :eof?
#force_quotes? ⇒ Boolean
(readonly)
Returns true
if all output fields are quoted. See .new for details.
# File 'lib/csv.rb', line 1117
def force_quotes? @writer_options[:force_quotes] end
#header_row? ⇒ Boolean
(readonly)
Returns true
if the next row read will be a header row.
# File 'lib/csv.rb', line 1299
def header_row? parser.header_row? end
#liberal_parsing? ⇒ Boolean
(readonly)
Returns true
if illegal input is handled. See .new for details.
# File 'lib/csv.rb', line 1122
def liberal_parsing? parser.liberal_parsing? end
#return_headers? ⇒ Boolean
(readonly)
Returns true
if headers will be returned as a row of results. See .new for details.
# File 'lib/csv.rb', line 1084
def return_headers? parser.return_headers? end
#skip_blanks? ⇒ Boolean
(readonly)
Returns true
blank lines are skipped by the parser. See .new for details.
# File 'lib/csv.rb', line 1112
def skip_blanks? parser.skip_blanks? end
#unconverted_fields? ⇒ Boolean
(readonly)
Returns true
if unconverted_fields() to parsed results. See .new for details.
# File 'lib/csv.rb', line 1060
def unconverted_fields? parser.unconverted_fields? end
#write_headers? ⇒ Boolean
(readonly)
Returns true
if headers are written in output. See .new for details.
# File 'lib/csv.rb', line 1092
def write_headers? @writer_options[:write_headers] end
Instance Method Details
#<<(row) Also known as: #add_row, #puts
The primary write method for wrapped Strings and IOs, row
(an ::Array
or ::CSV::Row
) is converted to CSV
and appended to the data source. When a ::CSV::Row
is passed, only the row’s fields() are appended to the output.
The data source must be open for writing.
# File 'lib/csv.rb', line 1229
def <<(row) writer << row self end
#add_row(row)
Alias for #<<.
# File 'lib/csv.rb', line 1233
alias_method :add_row, :<<
#build_fields_converter(initial_converters, options) (private)
[ GitHub ]# File 'lib/csv.rb', line 1460
def build_fields_converter(initial_converters, ) fields_converter = FieldsConverter.new( ) normalize_converters(initial_converters).each do |name, converter| fields_converter.add_converter(name, &converter) end fields_converter end
#build_header_fields_converter (private)
[ GitHub ]# File 'lib/csv.rb', line 1442
def build_header_fields_converter = { builtin_converters: HeaderConverters, accept_nil: true, } = @base_fields_converter_options.merge( ) build_fields_converter(@initial_header_converters, ) end
#build_parser_fields_converter (private)
[ GitHub ]# File 'lib/csv.rb', line 1430
def build_parser_fields_converter = { builtin_converters: Converters, } = @base_fields_converter_options.merge( ) build_fields_converter(@initial_converters, ) end
#build_writer_fields_converter (private)
[ GitHub ]# File 'lib/csv.rb', line 1455
def build_writer_fields_converter build_fields_converter(@initial_write_converters, @write_fields_converter_options) end
#col_sep
The encoded :col_sep
used in parsing and writing. See .new for details.
# File 'lib/csv.rb', line 1008
def col_sep parser.column_separator end
#convert(name)
#convert {|field| ... }
#convert {|field, field_info| ... }
You can use this method to install a Converters built-in, or provide a block that handles a custom conversion.
If you provide a block that takes one argument, it will be passed the field and is expected to return the converted value or the field itself. If your block takes two arguments, it will also be passed a ::CSV::FieldInfo
Struct, containing details about the field. Again, the block should return a converted field or the field itself.
# File 'lib/csv.rb', line 1251
def convert(name = nil, &converter) parser_fields_converter.add_converter(name, &converter) end
#convert_fields(fields, headers = false) (private)
# File 'lib/csv.rb', line 1405
def convert_fields(fields, headers = false) if headers header_fields_converter.convert(fields, nil, 0) else parser_fields_converter.convert(fields, @headers, lineno) end end
#converters
Returns the current list of converters in effect. See .new for details. Built-in converters will be returned by name, while others will be returned as is.
# File 'lib/csv.rb', line 1049
def converters parser_fields_converter.map do |converter| name = Converters.rassoc(converter) name ? name.first : converter end end
#determine_encoding(encoding, internal_encoding) (private)
[ GitHub ]# File 'lib/csv.rb', line 1368
def determine_encoding(encoding, internal_encoding) # honor the IO encoding if we can, otherwise default to ASCII-8BIT io_encoding = raw_encoding return io_encoding if io_encoding return Encoding.find(internal_encoding) if internal_encoding if encoding encoding, = encoding.split(":", 2) if encoding.is_a?(String) return Encoding.find(encoding) end Encoding.default_internal || Encoding.default_external end
#each(&block)
Yields each row of the data source in turn.
Support for Enumerable.
The data source must be open for reading.
# File 'lib/csv.rb', line 1279
def each(&block) parser_enumerator.each(&block) end
#eof? ⇒ Boolean
(readonly)
Also known as: #eof
# File 'lib/csv.rb', line 1197
def eof? return false if @eof_error begin parser_enumerator.peek false rescue MalformedCSVError => error @eof_error = error false rescue StopIteration true end end
#field_size_limit
The limit for field size, if any. See .new for details.
# File 'lib/csv.rb', line 1032
def field_size_limit parser.field_size_limit end
#flock(*args)
# File 'lib/csv.rb', line 1169
def flock(*args) raise NotImplementedError unless @io.respond_to?(:flock) @io.flock(*args) end
#gets
Alias for #shift.
# File 'lib/csv.rb', line 1321
alias_method :gets, :shift
#header_convert(name)
#header_convert {|field| ... }
#header_convert {|field, field_info| ... }
Identical to #convert(), but for header rows.
Note that this method must be called before header rows are read to have any effect.
# File 'lib/csv.rb', line 1266
def header_convert(name = nil, &converter) header_fields_converter.add_converter(name, &converter) end
#header_converters
Returns the current list of converters in effect for headers. See .new for details. Built-in converters will be returned by name, while others will be returned as is.
# File 'lib/csv.rb', line 1101
def header_converters header_fields_converter.map do |converter| name = HeaderConverters.rassoc(converter) name ? name.first : converter end end
#header_fields_converter (private)
[ GitHub ]# File 'lib/csv.rb', line 1438
def header_fields_converter @header_fields_converter ||= build_header_fields_converter end
#headers
Returns nil
if headers will not be used, true
if they will but have not yet been read, or the actual headers after they have been read. See .new for details.
# File 'lib/csv.rb', line 1069
def headers if @writer @writer.headers else parsed_headers = parser.headers return parsed_headers if parsed_headers raw_headers = @parser_options[:headers] raw_headers = nil if raw_headers == false raw_headers end end
#inspect
Returns a simplified description of the key CSV
attributes in an ASCII compatible ::String
.
# File 'lib/csv.rb', line 1328
def inspect str = ["#<", self.class.to_s, " io_type:"] # show type of wrapped IO if @io == $stdout then str << "$stdout" elsif @io == $stdin then str << "$stdin" elsif @io == $stderr then str << "$stderr" else str << @io.class.to_s end # show IO.path(), if available if @io.respond_to?(:path) and (p = @io.path) str << " io_path:" << p.inspect end # show encoding str << " encoding:" << @encoding.name # show other attributes ["lineno", "col_sep", "row_sep", "quote_char"].each do |attr_name| if a = __send__(attr_name) str << " " << attr_name << ":" << a.inspect end end ["skip_blanks", "liberal_parsing"].each do |attr_name| if a = __send__("#{attr_name}?") str << " " << attr_name << ":" << a.inspect end end _headers = headers str << " headers:" << _headers.inspect if _headers str << ">" begin str.join('') rescue # any encoding error str.map do |s| e = Encoding::Converter.asciicompat_encoding(s.encoding) e ? s.encode(e) : s.force_encoding("ASCII-8BIT") end.join('') end end
#ioctl(*args)
# File 'lib/csv.rb', line 1174
def ioctl(*args) raise NotImplementedError unless @io.respond_to?(:ioctl) @io.ioctl(*args) end
#line
The last row read from this file.
# File 'lib/csv.rb', line 1147
def line parser.line end
#lineno
The line number of the last row read from this file. Fields with nested line-end characters will not affect this count.
# File 'lib/csv.rb', line 1136
def lineno if @writer @writer.lineno else parser.lineno end end
#normalize_converters(converters) (private)
[ GitHub ]# File 'lib/csv.rb', line 1383
def normalize_converters(converters) converters ||= [] unless converters.is_a?(Array) converters = [converters] end converters.collect do |converter| case converter when Proc # custom code block [nil, converter] else # by name [converter, nil] end end end
#parser (private)
[ GitHub ]#parser_enumerator (private)
[ GitHub ]#parser_fields_converter (private)
[ GitHub ]# File 'lib/csv.rb', line 1426
def parser_fields_converter @parser_fields_converter ||= build_parser_fields_converter end
#parser_options (private)
[ GitHub ]# File 'lib/csv.rb', line 1472
def @parser_options.merge(header_fields_converter: header_fields_converter, fields_converter: parser_fields_converter) end
#path
[ GitHub ]# File 'lib/csv.rb', line 1179
def path @io.path if @io.respond_to?(:path) end
#puts(row)
Alias for #<<.
# File 'lib/csv.rb', line 1234
alias_method :puts, :<<
#quote_char
The encoded :quote_char
used in parsing and writing. See .new for details.
# File 'lib/csv.rb', line 1024
def quote_char parser.quote_character end
#raw_encoding (private)
Returns the encoding of the internal IO object.
#read Also known as: #readlines
Slurps the remaining rows and returns an ::Array
of Arrays.
The data source must be open for reading.
#readline
Alias for #shift.
# File 'lib/csv.rb', line 1322
alias_method :readline, :shift
#readlines
Alias for #read.
# File 'lib/csv.rb', line 1296
alias_method :readlines, :read
#rewind
Rewinds the underlying IO object and resets CSV’s lineno() counter.
# File 'lib/csv.rb', line 1212
def rewind @parser = nil @parser_enumerator = nil @eof_error = nil @writer.rewind if @writer @io.rewind end
#row_sep
The encoded :row_sep
used in parsing and writing. See .new for details.
# File 'lib/csv.rb', line 1016
def row_sep parser.row_separator end
#shift Also known as: #gets, #readline
The primary read method for wrapped Strings and IOs, a single row is pulled from the data source, parsed and returned as an ::Array
of fields (if header rows are not used) or a ::CSV::Row
(when header rows are used).
The data source must be open for reading.
# File 'lib/csv.rb', line 1310
def shift if @eof_error eof_error, @eof_error = @eof_error, nil raise eof_error end begin parser_enumerator.next rescue StopIteration nil end end
#skip_lines
The regex marking a line as a comment. See .new for details.
# File 'lib/csv.rb', line 1040
def skip_lines parser.skip_lines end
#stat(*args)
# File 'lib/csv.rb', line 1183
def stat(*args) raise NotImplementedError unless @io.respond_to?(:stat) @io.stat(*args) end
#to_i
# File 'lib/csv.rb', line 1188
def to_i raise NotImplementedError unless @io.respond_to?(:to_i) @io.to_i end
#to_io
[ GitHub ]# File 'lib/csv.rb', line 1193
def to_io @io.respond_to?(:to_io) ? @io.to_io : @io end
#writer (private)
[ GitHub ]#writer_fields_converter (private)
[ GitHub ]# File 'lib/csv.rb', line 1451
def writer_fields_converter @writer_fields_converter ||= build_writer_fields_converter end
#writer_options (private)
[ GitHub ]# File 'lib/csv.rb', line 1485
def @writer_options.merge(header_fields_converter: header_fields_converter, fields_converter: writer_fields_converter) end