Class: Daru::Vector

Inherits:
Object show all
Extended by:
Gem::Deprecate
Includes:
Maths::Arithmetic::Vector, Maths::Statistics::Vector, Enumerable
Defined in:
lib/daru/vector.rb,
lib/daru/extensions/rserve.rb

Overview

rubocop:disable Metrics/ClassLength

Constant Summary collapse

DEFAULT_SORTER =
lambda { |(lv, li), (rv, ri)|
  case
  when lv.nil? && rv.nil?
    li <=> ri
  when lv.nil?
    -1
  when rv.nil?
    1
  else
    lv <=> rv
  end
}
DATE_REGEXP =
/^(\d{2}-\d{2}-\d{4}|\d{4}-\d{2}-\d{2})$/

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Methods included from Maths::Statistics::Vector

#acf, #acvf, #average_deviation_population, #box_cox_transformation, #center, #coefficient_of_variation, #count, #covariance_population, #covariance_sample, #cumsum, #describe, #dichotomize, #diff, #ema, #emsd, #emv, #factors, #frequencies, #index_of_max, #index_of_min, #kurtosis, #macd, #max, #max_index, #mean, #median, #median_absolute_deviation, #min, #mode, #percent_change, #percentile, #product, #proportion, #proportions, #range, #ranked, #rolling, #rolling_count, #rolling_max, #rolling_mean, #rolling_median, #rolling_min, #rolling_std, #rolling_sum, #rolling_variance, #sample_with_replacement, #sample_without_replacement, #skew, #standard_deviation_population, #standard_deviation_sample, #standard_error, #standardize, #sum, #sum_of_squared_deviation, #sum_of_squares, #value_counts, #variance_population, #variance_sample, #vector_centered_compute, #vector_percentile, #vector_standardized_compute

Methods included from Maths::Arithmetic::Vector

#%, #*, #**, #+, #-, #/, #abs, #exp, #round, #sqrt

Constructor Details

#initialize(source, opts = {}) ⇒ Vector

Create a Vector object.

Arguments

Hash. If Array, a numeric index will be created if not supplied in the options. Specifying more index elements than actual values in source will insert nil into the surplus index elements. When a Hash is specified, the keys of the Hash are taken as the index elements and the corresponding values as the values that populate the vector.

Options

  • :name - Name of the vector

  • :index - Index of the vector

  • :dtype - The underlying data type. Can be :array, :nmatrix or :gsl.

Default :array.

  • :nm_dtype - For NMatrix, the data type of the numbers. See the NMatrix docs for

further information on supported data type.

  • :missing_values - An Array of the values that are to be treated as ‘missing’.

nil is the default missing value.

Usage

vecarr = Daru::Vector.new [1,2,3,4], index: [:a, :e, :i, :o]
vechsh = Daru::Vector.new({a: 1, e: 2, i: 3, o: 4})

Parameters:

  • source (Array, Hash)
    • Supply elements in the form of an Array or a



178
179
180
181
182
183
184
185
186
187
# File 'lib/daru/vector.rb', line 178

def initialize source, opts={}
  if opts[:type] == :category
    # Initialize category type vector
    extend Daru::Category
    initialize_category source, opts
  else
    # Initialize non-category type vector
    initialize_vector source, opts
  end
end

Dynamic Method Handling

This class handles dynamic methods through the method_missing method

#method_missing(name, *args, &block) ⇒ Object



1350
1351
1352
1353
1354
1355
1356
1357
1358
1359
1360
# File 'lib/daru/vector.rb', line 1350

def method_missing(name, *args, &block)
  # FIXME: it is shamefully fragile. Should be either made stronger
  # (string/symbol dychotomy, informative errors) or removed totally. - zverok
  if name =~ /(.+)\=/
    self[$1.to_sym] = args[0]
  elsif has_index?(name)
    self[name]
  else
    super
  end
end

Instance Attribute Details

#dataObject (readonly)

Store vector data in an array



142
143
144
# File 'lib/daru/vector.rb', line 142

def data
  @data
end

#dtypeObject (readonly)

The underlying dtype of the Vector. Can be either :array, :nmatrix or :gsl.



130
131
132
# File 'lib/daru/vector.rb', line 130

def dtype
  @dtype
end

#indexObject

The row index. Can be either Daru::Index or Daru::MultiIndex.



128
129
130
# File 'lib/daru/vector.rb', line 128

def index
  @index
end

#labelsObject

Store a hash of labels for values. Supplementary only. Recommend using index for proper usage.



140
141
142
# File 'lib/daru/vector.rb', line 140

def labels
  @labels
end

#missing_positionsObject (readonly)

An Array or the positions in the vector that are being treated as ‘missing’.



136
137
138
# File 'lib/daru/vector.rb', line 136

def missing_positions
  @missing_positions
end

#nameObject (readonly)

The name of the Daru::Vector. String.



126
127
128
# File 'lib/daru/vector.rb', line 126

def name
  @name
end

#nm_dtypeObject (readonly)

If the dtype is :nmatrix, this attribute represents the data type of the underlying NMatrix object. See NMatrix docs for more details on NMatrix data types.



134
135
136
# File 'lib/daru/vector.rb', line 134

def nm_dtype
  @nm_dtype
end

#plotting_libraryObject

Ploting library being used for this vector



144
145
146
# File 'lib/daru/vector.rb', line 144

def plotting_library
  @plotting_library
end

Class Method Details

.[](*indexes) ⇒ Object

Create a vector using (almost) any object

  • Array: flattened

  • Range: transformed using to_a

  • Daru::Vector

  • Numeric and string values

Description

The ‘Vector.[]` class method creates a vector from almost any object that has a `#to_a` method defined on it. It is similar to R’s ‘c` method.

Usage

a = Daru::Vector[1,2,3,4,6..10]
#=>
# <Daru::Vector:99448510 @name = nil @size = 9 >
#   nil
# 0   1
# 1   2
# 2   3
# 3   4
# 4   6
# 5   7
# 6   8
# 7   9
# 8  10


66
67
68
69
70
71
# File 'lib/daru/vector.rb', line 66

def [](*indexes)
  values = indexes.map do |a|
    a.respond_to?(:to_a) ? a.to_a : a
  end.flatten
  Daru::Vector.new(values)
end

._load(data) ⇒ Object

:nodoc:



73
74
75
76
77
78
79
# File 'lib/daru/vector.rb', line 73

def _load(data) # :nodoc:
  h = Marshal.load(data)
  Daru::Vector.new(h[:data],
    index: h[:index],
    name: h[:name],
    dtype: h[:dtype], missing_values: h[:missing_values])
end

.coerce(data, options = {}) ⇒ Object



81
82
83
84
85
86
87
88
89
90
# File 'lib/daru/vector.rb', line 81

def coerce(data, options={})
  case data
  when Daru::Vector
    data
  when Array, Hash
    new(data, options)
  else
    raise ArgumentError, "Can't coerce #{data.class} to #{self}"
  end
end

.new_with_size(n, opts = {}, &block) ⇒ Object

Create a new vector by specifying the size and an optional value and block to generate values.

Description

The new_with_size class method lets you create a Daru::Vector by specifying the size as the argument. The optional block, if supplied, is run once for populating each element in the Vector.

The result of each run of the block is the value that is ultimately assigned to that position in the Vector.

Options

:value All the rest like .new



33
34
35
36
37
# File 'lib/daru/vector.rb', line 33

def new_with_size n, opts={}, &block
  value = opts.delete :value
  block ||= ->(_) { value }
  Daru::Vector.new Array.new(n, &block), opts
end

Instance Method Details

#==(other) ⇒ Object

Two vectors are equal if they have the exact same index values corresponding with the exact same elements. Name is ignored.



298
299
300
301
302
303
304
305
306
# File 'lib/daru/vector.rb', line 298

def == other
  case other
  when Daru::Vector
    @index == other.index && size == other.size &&
      @index.all? { |index| self[index] == other[index] }
  else
    super
  end
end

#[](*input_indexes) ⇒ Object

Get one or more elements with specified index or a range.

Usage

# For vectors employing single layer Index

v[:one, :two] # => Daru::Vector with indexes :one and :two
v[:one]       # => Single element
v[:one..:three] # => Daru::Vector with indexes :one, :two and :three

# For vectors employing hierarchial multi index


215
216
217
218
219
220
221
222
223
224
225
226
227
228
# File 'lib/daru/vector.rb', line 215

def [](*input_indexes)
  # Get array of positions indexes
  positions = @index.pos(*input_indexes)

  # If one object is asked return it
  return @data[positions] if positions.is_a? Numeric

  # Form a new Vector using positional indexes
  Daru::Vector.new(
    positions.map { |loc| @data[loc] },
    name: @name,
    index: @index.subset(*input_indexes), dtype: @dtype
  )
end

#[]=(*indexes, val) ⇒ Object

Just like in Hashes, you can specify the index label of the Daru::Vector and assign an element an that place in the Daru::Vector.

Usage

v = Daru::Vector.new([1,2,3], index: [:a, :b, :c])
v[:a] = 999
#=>
##<Daru::Vector:90257920 @name = nil @size = 3 >
#    nil
#  a 999
#  b   2
#  c   3


286
287
288
289
290
291
292
293
294
# File 'lib/daru/vector.rb', line 286

def []=(*indexes, val)
  cast(dtype: :array) if val.nil? && dtype != :array

  guard_type_check(val)

  modify_vector(indexes, val)

  update_position_cache
end

#_dumpObject

:nodoc:



1320
1321
1322
1323
1324
1325
1326
1327
# File 'lib/daru/vector.rb', line 1320

def _dump(*) # :nodoc:
  Marshal.dump(
    data:           @data.to_a,
    dtype:          @dtype,
    name:           @name,
    index:          @index
  )
end

#all?(&block) ⇒ Boolean

Returns:

  • (Boolean)


574
575
576
# File 'lib/daru/vector.rb', line 574

def all? &block
  @data.data.all?(&block)
end

#any?(&block) ⇒ Boolean

Returns:

  • (Boolean)


570
571
572
# File 'lib/daru/vector.rb', line 570

def any? &block
  @data.data.any?(&block)
end

#at(*positions) ⇒ object

Returns vector of values given positional values

Examples:

dv = Daru::Vector.new 'a'..'e'
dv.at 0, 1, 2
# => #<Daru::Vector(3)>
#   0   a
#   1   b
#   2   c

Parameters:

  • *positions (Array<object>)

    positional values

Returns:

  • (object)

    vector



240
241
242
243
244
245
246
247
248
249
250
251
252
# File 'lib/daru/vector.rb', line 240

def at *positions
  # to be used to form index
  original_positions = positions
  positions = coerce_positions(*positions)
  validate_positions(*positions)

  if positions.is_a? Integer
    @data[positions]
  else
    values = positions.map { |pos| @data[pos] }
    Daru::Vector.new values, index: @index.at(*original_positions), dtype: dtype
  end
end

#bootstrap(estimators, nr, s = nil) ⇒ Object

Bootstrap

Generate nr resamples (with replacement) of size s from vector, computing each estimate from estimators over each resample. estimators could be a) Hash with variable names as keys and lambdas as values

a.bootstrap(:log_s2=>lambda {|v| Math.log(v.variance)},1000)

b) Array with names of method to bootstrap

a.bootstrap([:mean, :sd],1000)

c) A single method to bootstrap

a.jacknife(:mean, 1000)

If s is nil, is set to vector size by default.

Returns a DataFrame where each vector is a vector of length nr containing the computed resample estimates.



1119
1120
1121
1122
1123
1124
1125
1126
1127
1128
1129
1130
1131
1132
1133
1134
1135
# File 'lib/daru/vector.rb', line 1119

def bootstrap(estimators, nr, s=nil)
  s ||= size
  h_est, es, bss = prepare_bootstrap(estimators)

  nr.times do
    bs = sample_with_replacement(s)
    es.each do |estimator|
      bss[estimator].push(h_est[estimator].call(bs))
    end
  end

  es.each do |est|
    bss[est] = Daru::Vector.new bss[est]
  end

  Daru::DataFrame.new bss
end

#cast(opts = {}) ⇒ Object

Cast a vector to a new data type.

Options

  • :dtype - :array for Ruby Array. :nmatrix for NMatrix.

Raises:

  • (ArgumentError)


502
503
504
505
506
507
# File 'lib/daru/vector.rb', line 502

def cast opts={}
  dt = opts[:dtype]
  raise ArgumentError, "Unsupported dtype #{opts[:dtype]}" unless %i[array nmatrix gsl].include?(dt)

  @data = cast_vector_to dt unless @dtype == dt
end

#category?true, false

Tells if vector is categorical or not.

Examples:

dv = Daru::Vector.new [1, 2, 3], type: :category
dv.category?
# => true

Returns:

  • (true, false)

    true if vector is of type category, false otherwise



550
551
552
# File 'lib/daru/vector.rb', line 550

def category?
  type == :category
end

#clone_structureObject

Copies the structure of the vector (i.e the index, size, etc.) and fills all all values with nils.



1307
1308
1309
# File 'lib/daru/vector.rb', line 1307

def clone_structure
  Daru::Vector.new(([nil]*size), name: @name, index: @index.dup)
end

#concat(element, index) ⇒ Object Also known as: push, <<

Append an element to the vector by specifying the element and index

Raises:

  • (IndexError)


486
487
488
489
490
491
492
493
# File 'lib/daru/vector.rb', line 486

def concat element, index
  raise IndexError, 'Expected new unique index' if @index.include? index

  @index |= [index]
  @data[@index[index]] = element

  update_position_cache
end

#count_values(*values) ⇒ Integer

Count the number of values specified

Examples:

dv = Daru::Vector.new [1, 2, 1, 2, 3, 4, nil, nil]
dv.count_values nil
# => 2

Parameters:

  • *values (Array)

    values to count for

Returns:

  • (Integer)

    the number of times the values mentioned occurs



827
828
829
# File 'lib/daru/vector.rb', line 827

def count_values(*values)
  positions(*values).size
end

#cut(partitions, opts = {}) ⇒ Daru::Vector

Partition a numeric variable into categories.

Examples:

heights = Daru::Vector.new [30, 35, 32, 50, 42, 51]
height_cat = heights.cut [30, 40, 50, 60], labels=['low', 'medium', 'high']
# => #<Daru::Vector(6)>
#       0    low
#       1    low
#       2    low
#       3   high
#       4 medium
#       5   high

Parameters:

  • partitions (Array<Numeric>)

    an array whose consecutive elements provide intervals for categories

  • opts (Hash) (defaults to: {})

    options to cut the partition

Options Hash (opts):

  • :close_at (:left, :right)

    specifies whether the interval closes at the right side of left side

  • :labels (Array)

    names of the categories

Returns:

  • (Daru::Vector)

    numeric variable converted to categorical variable



1384
1385
1386
1387
1388
1389
1390
1391
1392
1393
1394
1395
1396
1397
1398
1399
1400
1401
# File 'lib/daru/vector.rb', line 1384

def cut partitions, opts={}
  close_at, labels = opts[:close_at] || :right, opts[:labels]
  partitions = partitions.to_a
  values = to_a.map { |val| cut_find_category partitions, val, close_at }
  cats = cut_categories(partitions, close_at)

  dv = Daru::Vector.new values,
    index: @index,
    type: :category,
    categories: cats

  # Rename categories if new labels provided
  if labels
    dv.rename_categories Hash[cats.zip(labels)]
  else
    dv
  end
end

#daru_vectorObject Also known as: dv

:nocov:



1330
1331
1332
# File 'lib/daru/vector.rb', line 1330

def daru_vector(*)
  self
end

#db_typeObject

Returns the database type for the vector, according to its content



1291
1292
1293
1294
1295
1296
1297
1298
1299
1300
1301
1302
1303
# File 'lib/daru/vector.rb', line 1291

def db_type
  # first, detect any character not number
  case
  when @data.any? { |v| v.to_s =~ DATE_REGEXP }
    'DATE'
  when @data.any? { |v| v.to_s =~ /[^0-9e.-]/ }
    'VARCHAR (255)'
  when @data.any? { |v| v.to_s =~ /\./ }
    'DOUBLE'
  else
    'INTEGER'
  end
end

#delete(element) ⇒ Object

Delete an element by value



510
511
512
# File 'lib/daru/vector.rb', line 510

def delete element
  delete_at index_of(element)
end

#delete_at(index) ⇒ Object

Delete element by index



515
516
517
518
519
520
# File 'lib/daru/vector.rb', line 515

def delete_at index
  @data.delete_at @index[index]
  @index = Daru::Index.new(@index.to_a - [index])

  update_position_cache
end

#delete_ifObject

Delete an element if block returns true. Destructive.



668
669
670
671
672
673
674
675
676
677
678
679
# File 'lib/daru/vector.rb', line 668

def delete_if
  return to_enum(:delete_if) unless block_given?

  keep_e, keep_i = each_with_index.reject { |n, _i| yield(n) }.transpose

  @data = cast_vector_to @dtype, keep_e
  @index = Daru::Index.new(keep_i)

  update_position_cache

  self
end

#detach_indexObject



802
803
804
805
806
807
# File 'lib/daru/vector.rb', line 802

def detach_index
  Daru::DataFrame.new(
    index: @index.to_a,
    values: @data.to_a
  )
end

#dupDaru::Vector

Duplicated a vector

Returns:



1100
1101
1102
# File 'lib/daru/vector.rb', line 1100

def dup
  Daru::Vector.new @data.dup, name: @name, index: @index.dup
end

#each(&block) ⇒ Object



97
98
99
100
101
102
# File 'lib/daru/vector.rb', line 97

def each(&block)
  return to_enum(:each) unless block_given?

  @data.each(&block)
  self
end

#each_index(&block) ⇒ Object



104
105
106
107
108
109
# File 'lib/daru/vector.rb', line 104

def each_index(&block)
  return to_enum(:each_index) unless block_given?

  @index.each(&block)
  self
end

#each_with_index(&block) ⇒ Object



111
112
113
114
115
116
117
# File 'lib/daru/vector.rb', line 111

def each_with_index &block
  return to_enum(:each_with_index) unless block_given?

  @data.to_a.zip(@index.to_a).each(&block)

  self
end

#empty?Boolean

Returns:

  • (Boolean)


433
434
435
# File 'lib/daru/vector.rb', line 433

def empty?
  @index.empty?
end

#group_by(*args) ⇒ Object



1416
1417
1418
# File 'lib/daru/vector.rb', line 1416

def group_by(*args)
  to_df.group_by(*args)
end

#has_index?(index) ⇒ Boolean

Returns true if an index exists

Returns:

  • (Boolean)


832
833
834
# File 'lib/daru/vector.rb', line 832

def has_index? index
  @index.include? index
end

#has_missing_data?Boolean Also known as: flawed?

Reports whether missing data is present in the Vector.

Returns:

  • (Boolean)


446
447
448
# File 'lib/daru/vector.rb', line 446

def has_missing_data?
  !indexes(*Daru::MISSING_VALUES).empty?
end

#head(q = 10) ⇒ Object



424
425
426
# File 'lib/daru/vector.rb', line 424

def head q=10
  self[0..(q-1)]
end

#in(other) ⇒ Object

Comparator for checking if any of the elements in other exist in self.

Examples:

Usage of ‘in`.

vector = Daru::Vector.new([1,2,3,4,5])
vector.where(vector.in([3,5]))
#=>
##<Daru::Vector:82215960 @name = nil @size = 2 >
#    nil
#  2   3
#  4   5

Parameters:

  • other (Array, Daru::Vector)

    A collection which has elements that need to be checked for in self.



375
376
377
378
379
380
381
382
# File 'lib/daru/vector.rb', line 375

def in other
  other = Hash[other.zip(Array.new(other.size, 0))]
  Daru::Core::Query::BoolArray.new(
    @data.each_with_object([]) do |d, memo|
      memo << (other.key?(d) ? true : false)
    end
  )
end

#include_values?(*values) ⇒ true, false

Check if any one of mentioned values occur in the vector

Examples:

dv = Daru::Vector.new [1, 2, 3, 4, nil]
dv.include_values? nil, Float::NAN
# => true

Parameters:

  • *values (Array)

    values to check for

Returns:

  • (true, false)

    returns true if any one of specified values occur in the vector



461
462
463
# File 'lib/daru/vector.rb', line 461

def include_values?(*values)
  values.any? { |v| include_with_nan? @data, v }
end

#index_of(element) ⇒ Object

Get index of element



555
556
557
558
559
560
# File 'lib/daru/vector.rb', line 555

def index_of element
  case dtype
  when :array then @index.key(@data.index { |x| x.eql? element })
  else @index.key @data.index(element)
  end
end

#indexes(*values) ⇒ Array

Return indexes of values specified

Examples:

dv = Daru::Vector.new [1, 2, nil, Float::NAN], index: 11..14
dv.indexes nil, Float::NAN
# => [13, 14]

Parameters:

  • *values (Array)

    values to find indexes for

Returns:

  • (Array)

    array of indexes of values specified



1238
1239
1240
# File 'lib/daru/vector.rb', line 1238

def indexes(*values)
  index.to_a.values_at(*positions(*values))
end

#inspect(spacing = 20, threshold = 15) ⇒ Object

Over rides original inspect for pretty printing in irb



1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
# File 'lib/daru/vector.rb', line 1007

def inspect spacing=20, threshold=15
  row_headers = index.is_a?(MultiIndex) ? index.sparse_tuples : index.to_a

  "#<#{self.class}(#{size})#{':category' if category?}>\n" +
    Formatters::Table.format(
      to_a.lazy.map { |v| [v] },
      headers: @name && [@name],
      row_headers: row_headers,
      threshold: threshold,
      spacing: spacing
    )
end

#is_values(*values) ⇒ Daru::Vector

Note:

Do not use it to check for Float::NAN as Float::NAN == Float::NAN is false

Return vector of booleans with value at ith position is either true or false depending upon whether value at position i is equal to any of the values passed in the argument or not

Examples:

dv = Daru::Vector.new [1, 2, 3, 2, 1]
dv.is_values 1, 2
# => #<Daru::Vector(5)>
#     0  true
#     1  true
#     2 false
#     3  true
#     4  true

Parameters:

  • *values (Array)

    values to equate with

Returns:



481
482
483
# File 'lib/daru/vector.rb', line 481

def is_values(*values)
  Daru::Vector.new values.map { |v| eq(v) }.inject(:|)
end

#jackknife(estimators, k = 1) ⇒ Object

Jacknife

Returns a dataset with jacknife delete-k estimators estimators could be: a) Hash with variable names as keys and lambdas as values

a.jacknife(:log_s2=>lambda {|v| Math.log(v.variance)})

b) Array with method names to jacknife

a.jacknife([:mean, :sd])

c) A single method to jacknife

a.jacknife(:mean)

k represent the block size for block jacknife. By default is set to 1, for classic delete-one jacknife.

Returns a dataset where each vector is an vector of length cases/k containing the computed jacknife estimates.

Reference:

  • Sawyer, S. (2005). Resampling Data: Using a Statistical Jacknife.



1154
1155
1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
1174
1175
1176
1177
1178
1179
# File 'lib/daru/vector.rb', line 1154

def jackknife(estimators, k=1) # rubocop:disable Metrics/AbcSize,Metrics/MethodLength
  raise "n should be divisible by k:#{k}" unless (size % k).zero?

  nb = (size / k).to_i
  h_est, es, ps = prepare_bootstrap(estimators)

  est_n = es.map { |v| [v, h_est[v].call(self)] }.to_h

  nb.times do |i|
    other = @data.dup
    other.slice!(i*k, k)
    other = Daru::Vector.new other

    es.each do |estimator|
      # Add pseudovalue
      ps[estimator].push(
        nb * est_n[estimator] - (nb-1) * h_est[estimator].call(other)
      )
    end
  end

  es.each do |est|
    ps[est] = Daru::Vector.new ps[est]
  end
  Daru::DataFrame.new ps
end

#keep_ifObject

Keep an element if block returns true. Destructive.



682
683
684
685
686
# File 'lib/daru/vector.rb', line 682

def keep_if
  return to_enum(:keep_if) unless block_given?

  delete_if { |val| !yield(val) }
end

#lag(k = 1) ⇒ Daru::Vector

Lags the series by ‘k` periods.

Lags the series by ‘k` periods, “shifting” data and inserting `nil`s from beginning or end of a vector, while preserving original vector’s size.

‘k` can be positive or negative integer. If `k` is positive, `nil`s are inserted at the beginning of the vector, otherwise they are inserted at the end.

Examples:

Lag a vector with different periods ‘k`


ts = Daru::Vector.new(1..5)
            # => [1, 2, 3, 4, 5]

ts.lag      # => [nil, 1, 2, 3, 4]
ts.lag(1)   # => [nil, 1, 2, 3, 4]
ts.lag(2)   # => [nil, nil, 1, 2, 3]
ts.lag(-1)  # => [2, 3, 4, 5, nil]

Parameters:

  • k (Integer) (defaults to: 1)

    “shift” the series by ‘k` periods. `k` can be positive or negative. (default = 1)

Returns:

  • (Daru::Vector)

    a new vector with “shifted” inital values and ‘nil` values inserted. The return vector is the same length as the orignal vector.



790
791
792
793
794
795
796
797
798
799
800
# File 'lib/daru/vector.rb', line 790

def lag k=1
  case k
  when 0 then dup
  when 1...size
    copy([nil] * k + data.to_a)
  when -size..-1
    copy(data.to_a[k.abs...size])
  else
    copy([])
  end
end

#map!(&block) ⇒ Object



119
120
121
122
123
# File 'lib/daru/vector.rb', line 119

def map!(&block)
  return to_enum(:map!) unless block_given?
  @data.map!(&block)
  self
end

#n_validObject

number of non-missing elements



815
816
817
# File 'lib/daru/vector.rb', line 815

def n_valid
  size - indexes(*Daru::MISSING_VALUES).size
end

#numeric?Boolean

Returns:

  • (Boolean)


437
438
439
# File 'lib/daru/vector.rb', line 437

def numeric?
  type == :numeric
end

#numeric_summaryString

Displays summary for an numeric type Vector

Returns:

  • (String)

    String containing numeric vector summary



991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
# File 'lib/daru/vector.rb', line 991

def numeric_summary
  summary = "\n  median: #{median}" +
            "\n  mean: %0.4f" % mean
  if sd
    summary << "\n  std.dev.: %0.4f" % sd +
               "\n  std.err.: %0.4f" % se
  end

  if count_values(*Daru::MISSING_VALUES).zero?
    summary << "\n  skew: %0.4f" % skew +
               "\n  kurtosis: %0.4f" % kurtosis
  end
  summary
end

#object?Boolean

Returns:

  • (Boolean)


441
442
443
# File 'lib/daru/vector.rb', line 441

def object?
  type == :object
end

#object_summaryString

Displays summary for an object type Vector

Returns:

  • (String)

    String containing object vector summary



976
977
978
979
980
981
982
983
984
985
986
987
# File 'lib/daru/vector.rb', line 976

def object_summary
  nval = count_values(*Daru::MISSING_VALUES)
  summary = "\n  factors: #{factors.to_a.join(',')}" \
            "\n  mode: #{mode.to_a.join(',')}" \
            "\n  Distribution\n"

  data = frequencies.sort.each_with_index.map do |v, k|
    [k, v, '%0.2f%%' % ((nval.zero? ? 1 : v.quo(nval))*100)]
  end

  summary + Formatters::Table.format(data)
end

#only_missing(as_a = :vector) ⇒ Object

Returns a Vector containing only missing data (preserves indexes).



1268
1269
1270
1271
1272
1273
1274
# File 'lib/daru/vector.rb', line 1268

def only_missing as_a=:vector
  if as_a == :vector
    self[*indexes(*Daru::MISSING_VALUES)]
  elsif as_a == :array
    self[*indexes(*Daru::MISSING_VALUES)].to_a
  end
end

#only_numericsObject

Returns a Vector with only numerical data. Missing data is included but non-Numeric objects are excluded. Preserves index.



1279
1280
1281
1282
1283
1284
1285
1286
# File 'lib/daru/vector.rb', line 1279

def only_numerics
  numeric_indexes =
    each_with_index
    .select { |v, _i| v.is_a?(Numeric) || v.nil? }
    .map(&:last)

  self[*numeric_indexes]
end

#only_valid(as_a = :vector, _duplicate = true) ⇒ Object

Creates a new vector consisting only of non-nil data

Arguments

as an Array. Otherwise will return a Daru::Vector.

vector, setting this to false will return the same vector. Otherwise, a duplicate will be returned irrespective of presence of missing data.



1193
1194
1195
1196
1197
1198
1199
1200
1201
1202
1203
1204
1205
1206
# File 'lib/daru/vector.rb', line 1193

def only_valid as_a=:vector, _duplicate=true
  # FIXME: Now duplicate is just ignored.
  #   There are no spec that fail on this case, so I'll leave it
  #   this way for now - zverok, 2016-05-07

  new_index = @index.to_a - indexes(*Daru::MISSING_VALUES)
  new_vector = new_index.map { |idx| self[idx] }

  if as_a == :vector
    Daru::Vector.new new_vector, index: new_index, name: @name, dtype: dtype
  else
    new_vector
  end
end

#positions(*values) ⇒ Object



1403
1404
1405
1406
1407
1408
1409
1410
1411
1412
1413
1414
# File 'lib/daru/vector.rb', line 1403

def positions(*values)
  case values
  when [nil]
    nil_positions
  when [Float::NAN]
    nan_positions
  when [nil, Float::NAN], [Float::NAN, nil]
    nil_positions + nan_positions
  else
    size.times.select { |i| include_with_nan? values, @data[i] }
  end
end

#recode(dt = nil, &block) ⇒ Object

Like map, but returns a Daru::Vector with the returned values.



652
653
654
655
656
# File 'lib/daru/vector.rb', line 652

def recode dt=nil, &block
  return to_enum(:recode) unless block_given?

  dup.recode! dt, &block
end

#recode!(dt = nil, &block) ⇒ Object

Destructive version of recode!



659
660
661
662
663
664
665
# File 'lib/daru/vector.rb', line 659

def recode! dt=nil, &block
  return to_enum(:recode!) unless block_given?

  @data.map!(&block).data
  @data = cast_vector_to(dt || @dtype)
  self
end

#reindex(new_index) ⇒ Object

Create a new vector with a different index, and preserve the indexing of current elements.



1068
1069
1070
# File 'lib/daru/vector.rb', line 1068

def reindex new_index
  dup.reindex!(new_index)
end

#reindex!(new_index) ⇒ Daru::Vector

Note:

Unlike #reorder! which takes positions as input it takes index as an input to reorder the vector

Sets new index for vector. Preserves index->value correspondence. Sets nil for new index keys absent from original index.

Parameters:

Returns:



1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
# File 'lib/daru/vector.rb', line 1026

def reindex! new_index
  values = []
  each_with_index do |val, i|
    values[new_index[i]] = val if new_index.include?(i)
  end
  values.fill(nil, values.size, new_index.size - values.size)

  @data = cast_vector_to @dtype, values
  @index = new_index

  update_position_cache

  self
end

#reject_values(*values) ⇒ Daru::Vector

Return a vector with specified values removed

Examples:

dv = Daru::Vector.new [1, 2, nil, Float::NAN]
dv.reject_values nil, Float::NAN
# => #<Daru::Vector(2)>
#   0   1
#   1   2

Parameters:

  • *values (Array)

    values to reject from resultant vector

Returns:



1218
1219
1220
1221
1222
1223
1224
1225
1226
1227
1228
1229
# File 'lib/daru/vector.rb', line 1218

def reject_values(*values)
  resultant_pos = size.times.to_a - positions(*values)
  dv = at(*resultant_pos)
  # Handle the case when number of positions is 1
  # and hence #at doesn't return a vector
  if dv.is_a?(Daru::Vector)
    dv
  else
    pos = resultant_pos.first
    at(pos..pos)
  end
end

#rename(new_name) ⇒ Object Also known as: name=

Give the vector a new name

Parameters:

  • new_name (Symbol)

    The new name.



1091
1092
1093
1094
# File 'lib/daru/vector.rb', line 1091

def rename new_name
  @name = new_name
  self
end

#reorder(order) ⇒ Object

Non-destructive version of #reorder!



1062
1063
1064
# File 'lib/daru/vector.rb', line 1062

def reorder order
  dup.reorder! order
end

#reorder!(order) ⇒ Object

Note:

Unlike #reindex! which takes index as input, it takes positions as an input to reorder the vector

Reorder the vector with given positions

Examples:

dv = Daru::Vector.new [3, 2, 1], index: ['c', 'b', 'a']
dv.reorder! [2, 1, 0]
# => #<Daru::Vector(3)>
#   a   1
#   b   2
#   c   3

Parameters:

  • order (Array)

    the order to reorder the vector with

Returns:

  • reordered vector



1053
1054
1055
1056
1057
1058
1059
# File 'lib/daru/vector.rb', line 1053

def reorder! order
  @index = @index.reorder order
  data_array = order.map { |i| @data[i] }
  @data = cast_vector_to @dtype, data_array, @nm_dtype
  update_position_cache
  self
end

#replace_nils(replacement) ⇒ Object

Non-destructive version of #replace_nils!



810
811
812
# File 'lib/daru/vector.rb', line 810

def replace_nils replacement
  dup.replace_nils!(replacement)
end

#replace_nils!(replacement) ⇒ Object

Replace all nils in the vector with the value passed as an argument. Destructive. See #replace_nils for non-destructive version

Arguments

  • replacement - The value which should replace all nils



755
756
757
758
759
760
761
# File 'lib/daru/vector.rb', line 755

def replace_nils! replacement
  indexes(*Daru::MISSING_VALUES).each do |idx|
    self[idx] = replacement
  end

  self
end

#replace_values(old_values, new_value) ⇒ Daru::Vector

Note:

It performs the replace in place.

Replaces specified values with a new value

Examples:

dv = Daru::Vector.new [1, 2, :a, :b]
dv.replace_values [:a, :b], nil
dv
# =>
# #<Daru::Vector:19903200 @name = nil @metadata = {} @size = 4 >
#     nil
#   0   1
#   1   2
#   2 nil
#   3 nil

Parameters:

  • old_values (Array)

    array of values to replace

  • new_value (object)

    new value to replace with

Returns:

  • (Daru::Vector)

    Same vector itself with values replaced with new value



1259
1260
1261
1262
1263
1264
1265
# File 'lib/daru/vector.rb', line 1259

def replace_values(old_values, new_value)
  old_values = [old_values] unless old_values.is_a? Array
  size.times do |pos|
    set_at([pos], new_value) if include_with_nan? old_values, at(pos)
  end
  self
end

#reset_index!Object



744
745
746
747
# File 'lib/daru/vector.rb', line 744

def reset_index!
  @index = Daru::Index.new(Array.new(size) { |i| i })
  self
end

#respond_to_missing?(name, include_private = false) ⇒ Boolean

Returns:

  • (Boolean)


1362
1363
1364
# File 'lib/daru/vector.rb', line 1362

def respond_to_missing?(name, include_private=false)
  name.to_s.end_with?('=') || has_index?(name) || super
end

#save(filename) ⇒ Object

Save the vector to a file

Arguments

  • filename - Path of file where the vector is to be saved



1316
1317
1318
# File 'lib/daru/vector.rb', line 1316

def save filename
  Daru::IO.save self, filename
end

#set_at(positions, val) ⇒ Object

Change value at given positions

Examples:

dv = Daru::Vector.new 'a'..'e'
dv.set_at [0, 1], 'x'
dv
# => #<Daru::Vector(5)>
#   0   x
#   1   x
#   2   c
#   3   d
#   4   e

Parameters:

  • *positions (Array<object>)

    positional values

  • val (object)

    value to assign



267
268
269
270
271
# File 'lib/daru/vector.rb', line 267

def set_at positions, val
  validate_positions(*positions)
  positions.map { |pos| @data[pos] = val }
  update_position_cache
end

#sizeObject



93
94
95
# File 'lib/daru/vector.rb', line 93

def size
  @data.size
end

#sort(opts = {}, &block) ⇒ Object

Sorts a vector according to its values. If a block is specified, the contents will be evaluated and data will be swapped whenever the block evaluates to true. Defaults to ascending order sorting. Any missing values will be put at the end of the vector. Preserves indexing. Default sort algorithm is quick sort.

Options

  • :ascending - if false, will sort in descending order. Defaults to true.

  • :type - Specify the sorting algorithm. Only supports quick_sort for now.

Usage

v = Daru::Vector.new ["My first guitar", "jazz", "guitar"]
# Say you want to sort these strings by length.
v.sort(ascending: false) { |a,b| a.length <=> b.length }


594
595
596
597
598
599
600
601
602
603
# File 'lib/daru/vector.rb', line 594

def sort opts={}, &block
  opts = {ascending: true}.merge(opts)

  vector_index = resort_index(@data.each_with_index, opts, &block)
  vector, index = vector_index.transpose

  index = @index.reorder index

  Daru::Vector.new(vector, index: index, name: @name, dtype: @dtype)
end

#sort_by_index(opts = {}) ⇒ Vector

Sorts the vector according to it’s`Index` values. Defaults to ascending order sorting.

Examples:


dv = Daru::Vector.new [11, 13, 12], index: [23, 21, 22]
# Say you want to sort index in ascending order
dv.sort_by_index(ascending: true)
#=> Daru::Vector.new [13, 12, 11], index: [21, 22, 23]
# Say you want to sort index in descending order
dv.sort_by_index(ascending: false)
#=> Daru::Vector.new [11, 12, 13], index: [23, 22, 21]

Parameters:

  • opts (Hash) (defaults to: {})

    the options for sort_by_index method.

Options Hash (opts):

  • :ascending (Boolean)

    false, will sort ‘index` in descending order.

Returns:

  • (Vector)

    new sorted ‘Vector` according to the index values.



623
624
625
626
627
628
# File 'lib/daru/vector.rb', line 623

def sort_by_index opts={}
  opts = {ascending: true}.merge(opts)
  _, new_order = resort_index(@index.each_with_index, opts).transpose

  reorder new_order
end

#sorted_data(&block) ⇒ Object

Just sort the data and get an Array in return using Enumerable#sort. Non-destructive. :nocov:



646
647
648
# File 'lib/daru/vector.rb', line 646

def sorted_data &block
  @data.to_a.sort(&block)
end

#split_by_separator(sep = ',') ⇒ Object

Returns a hash of Vectors, defined by the different values defined on the fields Example:

a=Daru::Vector.new(["a,b","c,d","a,b"])
a.split_by_separator
=>  {"a"=>#<Daru::Vector:0x7f2dbcc09d88
      @data=[1, 0, 1]>,
     "b"=>#<Daru::Vector:0x7f2dbcc09c48
      @data=[1, 1, 0]>,
    "c"=>#<Daru::Vector:0x7f2dbcc09b08
      @data=[0, 1, 1]>}


727
728
729
730
731
732
733
734
735
736
# File 'lib/daru/vector.rb', line 727

def split_by_separator sep=','
  split_data = splitted sep
  split_data
    .flatten.uniq.compact.map do |key|
    [
      key,
      Daru::Vector.new(split_data.map { |v| split_value(key, v) })
    ]
  end.to_h
end

#split_by_separator_freq(sep = ',') ⇒ Object



738
739
740
741
742
# File 'lib/daru/vector.rb', line 738

def split_by_separator_freq(sep=',')
  split_by_separator(sep).map { |k, v|
    [k, v.map(&:to_i).inject(:+)]
  }.to_h
end

#splitted(sep = ',') ⇒ Object

Return an Array with the data splitted by a separator.

a=Daru::Vector.new(["a,b","c,d","a,b","d"])
a.splitted
  =>
[["a","b"],["c","d"],["a","b"],["d"]]


702
703
704
705
706
707
708
709
710
711
712
# File 'lib/daru/vector.rb', line 702

def splitted sep=','
  @data.map do |s|
    if s.nil?
      nil
    elsif s.respond_to? :split
      s.split sep
    else
      [s]
    end
  end
end

#summary(indent_level = 0) ⇒ String

Create a summary of the Vector

Examples:

dv = Daru::Vector.new [1, 2, 3]
puts dv.summary

# =
#   n :3
#   non-missing:3
#   median: 2
#   mean: 2.0000
#   std.dev.: 1.0000
#   std.err.: 0.5774
#   skew: 0.0000
#   kurtosis: -2.3333

Returns:

  • (String)

    String containing the summary of the Vector



960
961
962
963
964
965
966
967
968
969
970
971
972
# File 'lib/daru/vector.rb', line 960

def summary(indent_level=0)
  non_missing = size - count_values(*Daru::MISSING_VALUES)
  summary = '  =' * indent_level + "= #{name}" \
            "\n  n :#{size}" \
            "\n  non-missing:#{non_missing}"
  case type
  when :object
    summary << object_summary
  when :numeric
    summary << numeric_summary
  end
  summary.split("\n").join("\n" + '  ' * indent_level)
end

#tail(q = 10) ⇒ Object



428
429
430
431
# File 'lib/daru/vector.rb', line 428

def tail q=10
  start = [size - q, 0].max
  self[start..(size-1)]
end

#to_aObject

Return an array



899
900
901
# File 'lib/daru/vector.rb', line 899

def to_a
  @data.to_a
end

#to_category(opts = {}) ⇒ Daru::Vector

Converts a non category type vector to category type vector.

Parameters:

  • opts (Hash) (defaults to: {})

    options to convert to category

Options Hash (opts):

  • :ordered (true, false)

    Specify if vector is ordered or not. If it is ordered, it can be sorted and min, max like functions would work

  • :categories (Array)

    set categories in the specified order

Returns:



1343
1344
1345
1346
1347
1348
# File 'lib/daru/vector.rb', line 1343

def to_category opts={}
  dv = Daru::Vector.new to_a, type: :category, name: @name, index: @index
  dv.ordered = opts[:ordered] || false
  dv.categories = opts[:categories] if opts[:categories]
  dv
end

#to_dfDaru::DataFrame

Returns the vector as a single-vector dataframe.

Returns:



837
838
839
# File 'lib/daru/vector.rb', line 837

def to_df
  Daru::DataFrame.new({@name => @data}, name: @name, index: @index)
end

#to_gslObject

If dtype != gsl, will convert data to GSL::Vector with to_a. Otherwise returns the stored GSL::Vector object.

Raises:

  • (NoMethodError)


884
885
886
887
888
889
890
891
# File 'lib/daru/vector.rb', line 884

def to_gsl
  raise NoMethodError, 'Install gsl-nmatrix for access to this functionality.' unless Daru.has_gsl?
  if dtype == :gsl
    @data.data
  else
    GSL::Vector.alloc(reject_values(*Daru::MISSING_VALUES).to_a)
  end
end

#to_hObject

Convert to hash (explicit). Hash keys are indexes and values are the correspoding elements



894
895
896
# File 'lib/daru/vector.rb', line 894

def to_h
  @index.map { |index| [index, self[index]] }.to_h
end

#to_html(threshold = 30) ⇒ Object

Convert to html for iruby



909
910
911
912
913
914
915
916
917
918
# File 'lib/daru/vector.rb', line 909

def to_html(threshold=30)
  table_thead = to_html_thead
  table_tbody = to_html_tbody(threshold)
  path = if index.is_a?(MultiIndex)
           File.expand_path('../iruby/templates/vector_mi.html.erb', __FILE__)
         else
           File.expand_path('../iruby/templates/vector.html.erb', __FILE__)
         end
  ERB.new(File.read(path).strip).result(binding)
end

#to_html_tbody(threshold = 30) ⇒ Object



930
931
932
933
934
935
936
937
938
# File 'lib/daru/vector.rb', line 930

def to_html_tbody(threshold=30)
  table_tbody_path =
    if index.is_a?(MultiIndex)
      File.expand_path('../iruby/templates/vector_mi_tbody.html.erb', __FILE__)
    else
      File.expand_path('../iruby/templates/vector_tbody.html.erb', __FILE__)
    end
  ERB.new(File.read(table_tbody_path).strip).result(binding)
end

#to_html_theadObject



920
921
922
923
924
925
926
927
928
# File 'lib/daru/vector.rb', line 920

def to_html_thead
  table_thead_path =
    if index.is_a?(MultiIndex)
      File.expand_path('../iruby/templates/vector_mi_thead.html.erb', __FILE__)
    else
      File.expand_path('../iruby/templates/vector_thead.html.erb', __FILE__)
    end
  ERB.new(File.read(table_thead_path).strip).result(binding)
end

#to_jsonObject

Convert the hash from to_h to json



904
905
906
# File 'lib/daru/vector.rb', line 904

def to_json(*)
  to_h.to_json
end

#to_matrix(axis = :horizontal) ⇒ Object

Convert Vector to a horizontal or vertical Ruby Matrix.

Arguments

  • axis - Specify whether you want a :horizontal or a :vertical matrix.



846
847
848
849
850
851
852
853
854
# File 'lib/daru/vector.rb', line 846

def to_matrix axis=:horizontal
  if axis == :horizontal
    Matrix[to_a]
  elsif axis == :vertical
    Matrix.columns([to_a])
  else
    raise ArgumentError, "axis should be either :horizontal or :vertical, not #{axis}"
  end
end

#to_nmatrix(axis = :horizontal) ⇒ NMatrix

Convert vector to nmatrix object

Examples:

dv = Daru::Vector.new [1, 2, 3]
dv.to_nmatrix
# =>
# [
#   [1, 2, 3] ]

Parameters:

  • axis (Symbol) (defaults to: :horizontal)

    :horizontal or :vertical

Returns:

  • (NMatrix)

    NMatrix object containing all values of the vector



865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
# File 'lib/daru/vector.rb', line 865

def to_nmatrix axis=:horizontal
  unless numeric? && !include?(nil)
    raise ArgumentError, 'Can not convert to nmatrix'\
      'because the vector is numeric'
  end

  case axis
  when :horizontal
    NMatrix.new [1, size], to_a
  when :vertical
    NMatrix.new [size, 1], to_a
  else
    raise ArgumentError, 'Invalid axis specified. '\
      'Valid axis are :horizontal and :vertical'
  end
end

#to_REXPObject

rubocop:disable Style/MethodName



17
18
19
# File 'lib/daru/extensions/rserve.rb', line 17

def to_REXP # rubocop:disable Style/MethodName
  Rserve::REXP::Wrapper.wrap(to_a)
end

#to_sObject



940
941
942
# File 'lib/daru/vector.rb', line 940

def to_s
  "#<#{self.class}#{': ' + @name.to_s if @name}(#{size})#{':category' if category?}>"
end

#typeObject

The type of data contained in the vector. Can be :object or :numeric. If the underlying dtype is an NMatrix, this method will return the data type of the NMatrix object.

Running through the data to figure out the kind of data is delayed to the last possible moment.



528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
# File 'lib/daru/vector.rb', line 528

def type
  return @data.nm_dtype if dtype == :nmatrix

  if @type.nil? || @possibly_changed_type
    @type = :numeric
    each do |e|
      next if e.nil? || e.is_a?(Numeric)
      @type = :object
      break
    end
    @possibly_changed_type = false
  end

  @type
end

#uniqObject

Keep only unique elements of the vector alongwith their indexes.



563
564
565
566
567
568
# File 'lib/daru/vector.rb', line 563

def uniq
  uniq_vector = @data.uniq
  new_index   = uniq_vector.map { |element| index_of(element) }

  Daru::Vector.new uniq_vector, name: @name, index: new_index, dtype: @dtype
end

#verifyObject

Reports all values that doesn’t comply with a condition. Returns a hash with the index of data and the invalid data.



690
691
692
693
694
695
# File 'lib/daru/vector.rb', line 690

def verify
  (0...size)
    .map { |i| [i, @data[i]] }
    .reject { |_i, val| yield(val) }
    .to_h
end

#where(bool_array) ⇒ Object

Return a new vector based on the contents of a boolean array. Use with the comparator methods to obtain meaningful results. See this notebook for a good overview of using #where.

Parameters:

  • bool_arry (Daru::Core::Query::BoolArray, Array<TrueClass, FalseClass>)

    The collection containing the true of false values. Each element in the Vector corresponding to a ‘true` in the bool_arry will be returned alongwith it’s index.



420
421
422
# File 'lib/daru/vector.rb', line 420

def where bool_array
  Daru::Core::Query.vector_where self, bool_array
end