daru - Data Analysis in RUby

Gem Version Build Status

Introduction

daru (Data Analysis in RUby) is a library for storage, analysis, manipulation and visualization of data in Ruby.

daru makes it easy and intuituive to process data predominantly through 2 data structures: Daru::DataFrame and Daru::Vector. Written in pure Ruby works with all ruby implementations. Tested with MRI 2.0, 2.1, 2.2 and 2.3.

Features

  • Data structures:
    • Vector - A basic 1-D vector.
    • DataFrame - A 2-D spreadsheet-like structure for manipulating and storing data sets. This is daru's primary data structure.
  • Compatible with IRuby notebook, statsample, statsample-glm and statsample-timeseries.
  • Support for time series.
  • Singly and hierarchially indexed data structures.
  • Flexible and intuitive API for manipulation and analysis of data.
  • Easy plotting, statistics and arithmetic.
  • Plentiful iterators.
  • Optional speed and space optimization on MRI with NMatrix and GSL.
  • Easy splitting, aggregation and grouping of data.
  • Quickly reducing data with pivot tables for quick data summary.
  • Import and export data from and to Excel, CSV, SQL Databases, ActiveRecord and plain text files.

Notebooks

Notebooks on most use cases

Notebooks on Time series

Case Studies

Blog Posts

Time series

Basic Usage

daru exposes two major data structures: DataFrame and Vector. The Vector is a basic 1-D structure corresponding to a labelled Array, while the DataFrame - daru's primary data structure - is 2-D spreadsheet-like structure for manipulating and storing data sets.

Basic DataFrame intitialization.

data_frame = Daru::DataFrame.new(
  {
    'Beer' => ['Kingfisher', 'Snow', 'Bud Light', 'Tiger Beer', 'Budweiser'],
    'Gallons sold' => [500, 400, 450, 200, 250]
  },
  index: ['India', 'China', 'USA', 'Malaysia', 'Canada']
)
data_frame

init0

Load data from CSV files.

df = Daru::DataFrame.from_csv('TradeoffData.csv')

init1

Basic Data Manipulation

Selecting rows.

data_frame.row['USA']

man0

Selecting columns.

data_frame['Beer']

man1

A range of rows.

data_frame.row['India'..'USA']

man2

The first 2 rows.

data_frame.first(2)

man3

The last 2 rows.

data_frame.last(2)

man4

Adding a new column.

data_frame['Gallons produced'] = [550, 500, 600, 210, 240]

man5

Creating a new column based on data in other columns.

data_frame['Demand supply gap'] = data_frame['Gallons produced'] - data_frame['Gallons sold']

man6

Condition based selection

Selecting countries based on the number of gallons sold in each. We use a syntax similar to that defined by Arel, i.e. by using the where clause.

data_frame.where(data_frame['Gallons sold'].lt(300))

con0

You can pass a combination of boolean operations into the #where method and it should work fine:

data_frame.where(
  data_frame['Beer']
  .in(['Snow', 'Kingfisher','Tiger Beer'])
  .and(
    data_frame['Gallons produced'].gt(520).or(data_frame['Gallons produced'].lt(250))
  )
)

con1

Plotting

Daru supports plotting of interactive graphs with nyaplot. You can easily create a plot with the #plot method. Here we plot the gallons sold on the Y axis and name of the brand on the X axis in a bar graph.

data_frame.plot type: :bar, x: 'Beer', y: 'Gallons sold' do |plot, diagram|
  plot.x_label "Beer"
  plot.y_label "Gallons Sold"
  plot.yrange [0,600]
  plot.width 500
  plot.height 400
end

plot0

In addition to nyaplot, daru also supports plotting out of the box with gnuplotrb.

Documentation

Docs can be found here.

Roadmap

  • Enable creation of DataFrame by only specifying an NMatrix/MDArray in initialize. Vector naming happens automatically (alphabetic) or is specified in an Array.
  • Assignment of a column to a single number should set the entire column to that number.
  • Multiple column assignment with []=
  • Multiple value assignment for vectors with []=.
  • #find_max function which will evaluate a block and return the row for the value of the block is max.
  • Sort by index.
  • Statistics on DataFrame over rows.
  • Calculate percentage change.
  • Have some sample data sets for users to play around with. Should be able to load these from the code itself.
  • Sorting with missing data present.

Contributing

Pick a feature from the Roadmap or the issue tracker or think of your own and send me a Pull Request!

For details see CONTRIBUTING.

Acknowledgements

  • Google and the Ruby Science Foundation for the Google Summer of Code 2015 grant for further developing daru and integrating it with other ruby gems.
  • Thank you last.fm for making user data accessible to the public.

Copyright (c) 2015, Sameer Deshmukh All rights reserved