TinyTDS - A modern, simple and fast FreeTDS library for Ruby using DB-Library.

The TinyTDS gem is meant to serve the extremely common use-case of connecting, querying and iterating over results to Microsoft SQL Server databases from ruby. Even though it uses FreeTDS’s DB-Library, it is NOT meant to serve as direct 1:1 mapping of that complex C API.

The benefits are speed, automatic casting to ruby primitives, and proper encoding support. It converts all SQL Server datatypes to native ruby objects supporting :utc or :local time zones for time-like types. To date it is the only ruby client library that allows client encoding options, defaulting to UTF-8, while connecting to SQL Server. It also properly encodes all string and binary data. The motivation for TinyTDS is to become the de-facto low level connection mode for the SQL Server adapter for ActiveRecord. For further details see the special thanks section at the bottom

The API is simple and consists of these classes:

  • TinyTds::Client - Your connection to the database.

  • TinyTds::Result - Returned from issuing an #execute on the connection. It includes Enumerable.

  • TinyTds::Error - A wrapper for all exceptions.

New & Noteworthy

  • Works with FreeTDS 0.83.dev

  • Tested on Windows using MiniPortile & RailsInstaller.

Install

Installing with rubygems should just work. TinyTDS is tested on ruby version 1.8.6, 1.8.7, 1.9.1, 1.9.2 as well as REE & JRuby.

$ gem install tiny_tds

Although we search for FreeTDS’s libraries and headers, you may have to specify include and lib directories using “–with-freetds-include=/some/local/include/freetds” and “–with-freetds-lib=/some/local/lib”

FreeTDS Compatibility

TinyTDS is developed for FreeTDS 0.82 & 0.83.dev. It is tested with SQL Server 2000, 2005, and 2008 using TDS Version 8.0. We utilize FreeTDS’s db-lib client library. We compile against sybdb.h and define MSDBLIB which means that our client enables Microsoft behavior in the db-lib API where it diverges from Sybase’s. You do NOT need to compile FreeTDS with the “–enable-msdblib” option for our client to work properly. However, please make sure to compile FreeTDS with libiconv support for encodings to work at their best. Run “tsql -C” in your console and check for “iconv library: yes”.

Data Types

Our goal is to support every SQL Server data type and covert it to a logical ruby object. When dates or times are returned, they are instantiated to either :utc or :local time depending on the query options. Under ruby 1.9, all strings are encoded to the connection’s encoding and all binary data types are associated to ruby’s ASCII-8BIT/BINARY encoding.

Below is a list of the data types we plan to support using future versions of FreeTDS. They are associated with SQL Server 2008. All unsupported data types are returned as properly encoded strings.

  • date
  • datetime2
  • datetimeoffset
  • time

TinyTds::Client Usage

Connect to a database.

client = TinyTds::Client.new(:username => 'sa', :password => 'secret', :dataserver => 'mytds_box')

Creating a new client takes a hash of options. For valid iconv encoding options, see the output of “iconv -l”. Only a few have been tested and highly recommended to leave blank for the UTF-8 default.

  • :username - The database server user.

  • :password - The user password.

  • :dataserver - The name for your server as defined in freetds.conf.

  • :database - The default database to use.

  • :appname - Short string seen in SQL Servers process/activity window.

  • :tds_version - TDS version. Defaults to 80, not recommended to change.

  • :login_timeout - Seconds to wait for login. Default to 60 seconds.

  • :timeout - Seconds to wait for a response to a SQL command. Default 5 seconds.

  • :encoding - Any valid iconv value like CP1251 or ISO-8859-1. Default UTF-8.

Close and free a clients connection.

client.close
client.closed? # => true

Escape strings.

client.escape("How's It Going'") # => "How''s It Going''"

Send a SQL string to the database and return a TinyTds::Result object.

result = client.execute("SELECT * FROM [datatypes]")

TinyTds::Result Usage

A result object is returned by the client’s execute command. It is important that you either return the data from the query, most likely with the #each method, or that you cancel the results before asking the client to execute another SQL batch. Failing to do so will yield an error.

Calling #each on the result will lazily load each row from the database.

result.each do |row|
  # By default each row is a hash.
  # The keys are the fields, as you'd expect.
  # The values are pre-built ruby primitives mapped from their corresponding types.
  # Here's an leemer: http://is.gd/g61xo
end

A result object has a #fields accessor. It can be called before the result rows are iterated over. Even if no rows are returned, #fields will still return the column names you expected. Any SQL that does not return columned data will always return an empty array for #fields. It is important to remember that if you access the #fields before iterating over the results, the columns will always follow the default query option’s :symbolize_keys setting at the client’s level and will ignore the query options passed to each.

result = client.execute("USE [tinytds_test]")
result.fields # => []
result.do

result = client.execute("SELECT [id] FROM [datatypes]")
result.fields # => ["id"]
result.cancel
result = client.execute("SELECT [id] FROM [datatypes]")
result.each(:symbolize_keys => true)
result.fields # => [:id]

You can cancel a result object’s data from being loading by the server.

result = client.execute("SELECT * FROM [super_big_table]")
result.cancel

If the SQL executed by the client returns affected rows, you can easily find out how many.

result.each
result.affected_rows # => 24

This pattern is so common for UPDATE and DELETE statements that the #do method cancels any need for loading the result data and returns the #affected_rows.

result = client.execute("DELETE FROM [datatypes]")
result.do # => 72

Likewise for INSERT statements, the #insert method cancels any need for loading the result data and executes a SCOPE_IDENTITY() for the primary key.

result = client.execute("INSERT INTO [datatypes] ([xml]) VALUES ('<html><br/></html>')")
result.insert # => 420

The result object can handle multiple result sets form batched SQL or stored procedures. It is critical to remember that when calling each with a block for the first time will return each “row” of each result set. Calling each a second time with a block will yield each “set”.

sql = ["SELECT TOP (1) [id] FROM [datatypes]", 
       "SELECT TOP (2) [bigint] FROM [datatypes] WHERE [bigint] IS NOT NULL"].join(' ')

set1, set2 = client.execute(sql).each
set1 # => [{"id"=>11}]
set2 # => [{"bigint"=>-9223372036854775807}, {"bigint"=>9223372036854775806}]

result = client.execute(sql)

result.each do |rowset|
  # First time data loading, yields each row from each set.
  # 1st: {"id"=>11}
  # 2nd: {"bigint"=>-9223372036854775807}
  # 3rd: {"bigint"=>9223372036854775806}
end

result.each do |rowset|
  # Second time over (if columns cached), yields each set.
  # 1st: [{"id"=>11}]
  # 2nd: [{"bigint"=>-9223372036854775807}, {"bigint"=>9223372036854775806}]
end

Use the #sqlsent? and #canceled? query methods on the client to determine if an active SQL batch still needs to be processed and or if data results were canceled from the last result object. These values reset to true and false respectively for the client at the start of each #execute and new result object. Or if all rows are processed normally, #sqlsent? will return false. To demonstrate, lets assume we have 100 rows in the result object.

client.sqlsent?   # = false
client.canceled?  # = false

result = client.execute("SELECT * FROM [super_big_table]")

client.sqlsent?   # = true
client.canceled?  # = false

result.each do |row|
  # Assume we break after 20 rows with 80 still pending.
  break if row["id"] > 20
end

client.sqlsent?   # = true
client.canceled?  # = false

result.cancel

client.sqlsent?   # = false
client.canceled?  # = true

It is possible to get the return code after executing a stored procedure from either the result or client object.

client.return_code  # => nil

result = client.execute("EXEC tinytds_TestReturnCodes")
result.return_code  # => 420
client.return_code  # => 420

Query Options

Every TinyTds::Result object can pass query options to the #each method. The defaults are defined and configurable by setting options in the TinyTds::Client.default_query_options hash. The default values are:

  • :as => :hash - Object for each row yielded. Can be set to :array.

  • :symbolize_keys => false - Row hash keys. Defaults to shared/frozen string keys.

  • :cache_rows => true - Successive calls to #each returns the cached rows.

  • :timezone => :local - Local to the ruby client or :utc for UTC.

Each result gets a copy of the default options you specify at the client level and can be overridden by passing an options hash to the #each method. For example

result.each(:as => :array, :cache_rows => false) do |row|
  # Each row is now an array of values ordered by #fields.
  # Rows are yielded and forgotten about, freeing memory.
end

Besides the standard query options, the result object can take one additional option. Using :first => true will only load the first row of data and cancel all remaining results.

result = client.execute("SELECT * FROM [super_big_table]")
result.each(:first => true) # => [{'id' => 24}]

Row Caching

By default row caching is turned on because the SQL Server adapter for ActiveRecord would not work without it. I hope to find some time to create some performance patches for ActiveRecord that would allow it to take advantages of lazily created yielded rows from result objects. Currently only TinyTDS and the Mysql2 gem allow such a performance gain.

Using TinyTDS With Rails & The ActiveRecord SQL Server adapter.

As of version 2.3.11 & 3.0.3 of the adapter, you can specify a :dblib mode in database.yml and use TinyTDS as the low level connection mode. Make sure to add a :dataserver option to that matches the name in your freetds.conf file. The SQL Server adapter can be found using the link below. Also included is a direct link to the wiki article covering common questions when using TinyTDS as the low level connection mode for the adapter.

github.com/rails-sqlserver/activerecord-sqlserver-adapter/wiki/Using-TinyTds

Development & Testing

We use bundler for development. Simply run “bundle install” then “rake” to build the gem and run the unit tests. The tests assume you have created a database named “tinytds_test” accessible by a database owner named “tinytds”. Before running the test rake task, you may need to define a pair of environment variables that help the client connect to your specific FreeTDS database server name and which schema (2000, 2005 or 2008) to use. For example:

$ rake TINYTDS_UNIT_DATASERVER=mydbserver TINYTDS_SCHEMA=sqlserver_2008

If you are compiling locally using MiniPortile (the default dev setup) and you do not have a user based freetds.conf file with your dataservers, you may have to configure the locally compiled freetds.conf file. You can find it at the ports/<platform>/freetds/<version>/etc/freetds.conf path locally to your repo. In this situation you may have to set the FREETDS environment variable too this full path.

For help and support.

Current to do list.

  • Install an interrupt handler.

  • Allow #escape to accept all ruby primitives.

  • Get bug reports!

About Me

My name is Ken Collins and to avoid confusion – I have no love for Microsoft nor do I work on Windows or have I ever owned a PC, just so we know :) – I currently maintain the SQL Server adapter for ActiveRecord and wrote this library as my first cut into learning ruby C extensions. Hopefully it will help promote the power of ruby and the rails framework to those that have not yet discovered it. My blog is metaskills.net and I can be found on twitter as @metaskills. Enjoy!

Special Thanks