Module: CShadow

Defined in:
lib/cgen/cshadow.rb,
lib/cgen/attribute.rb

Overview

Overview

CShadow is a way of creating objects which transparently mix C data with Ruby data.

The CShadow module is a mix-in which adds a C struct to objects which derive from the class which includes it. The data members of the C struct, called shadow attributes, may vary in subclasses of this base class. Shadow attrs can be added to any class inheriting from the base class, and they will propagate along the ordinary ruby inheritance hierarchy. (For now, shadow attributes cannot be defined in modules.)

The CShadow module uses CGenerator’s structure templates to handle the code generation. CGenerator is also useful for inline C definitons of methods that operate on shadow attributes.

CShadow cooperates with the CShadow::Attribute subclasses defined in attribute.rb to handle the mark and free functions, type checking and conversion, serialization, and so on. New attribute types can be added easily.

Usage

class MyClass
  include CShadow
  shadow_attr_accessor :x => "int x" # fox example
end

Include CShadow in the base class(es) that need to have shadow attributes. The base class is assigned a CGenerator::Library, which can be accessed using the base class’s #shadow_library method. Each subclass of the base class will be associated with a struct defined in this library’s .h files.

As usual, the #initialize method of the class will be called when the object is created, and the arguments are whatever was passed to #new, including the block, if any. You can write your own #initialize method to assign initial values to the shadow attrs. (Typically, shadow attrs are initialized by default to nil or 0. See attribute.rb.)

The file name of the library is the same as the name of the class which includes CShadow, by default, and the library directory is generated in the current dir when #commit is called. To change the name, or to use a different library:

shadow_library <Library, Class, or String>

The argument can be another library (instance of CShadow::Library), a class which includes CShadow, in which case the library of the class is used, or a string, in which case a new library is created. Using this feature, several base classes (and their descendants) can be kept in the same library. It is not possible to split a tree (a base class which includes CShadow and its descendants) into more than one library.

Shadow classes that are placed in the same library can still be put in separate C source files, using #shadow_library_file:

shadow_library_file <CFile or String>

This setting is inherited (and can be overridden) by subclasses of the current class. If a class calls both #shadow_library and #shadow_library_file then they must be called in that order. Note that anything defined before the #shadow_library_file statement will not be placed in the specifed file.

The C include and source files for the current class can be accessed with #shadow_library_include_file and #shadow_library_source_file. This is not required for normal use.

Behavior at commit time can be controlled by scheduling #before_commit and #after_commit procs. These procs are called immediately before and after the actual commit, which allows, for example, removing instances that would otherwise cause an exception, or creating the instance of a singleton class. The #before_commit and #after_commit class methods of CShadow delegate to the shadow_library’s methods. See CGenerator for details.

Struct members

All shadow structs have a self member of type VALUE which points to the Ruby object.

The subclass struct inherits members of the base class struct.

There are two types of shadow attributes, declared with #shadow_attr and friends:

(1) C data

:d => “double d”, :x => “char *foo”

(2) Ruby value:

:str => String, :obj => Object

In addition to shadow attributes, you can declare other struct members using

shadow_struct.declare ...

as in cgen.

For example, the ruby code:

produces the following struct defs (as well as some functions):

Type checking and conversion

In case (1) (C data attribute), assignments to int and double struct members are done with NUM2INT and NUM2DBL, which convert between numeric types, but raise exceptions for unconvertible types. The char * case uses StringValue (formerly rb_str2cstr), which behaves analogously.

In case (2) (Ruby value attribute) the assigned value must always be a descendant of the specified type, except that nil is always accepted. The purpose of specifying a class (should also allow type predicate!) is to allow C code to assume that certain struct members are present in the shadow struct, and so it can be safely cast to the “ancestor” struct.

Adding methods

CGenerator provides a general interface to the Ruby-C api. However, for simplicity, CShadow defines three methods in the client class for defining methods and class methods:

CShadow.define_c_method CShadow.define_c_class_method CShadow.define_c_function

Memory management

Each type of attribute is responsible for managing any required marking of referenced Ruby objects or freeing of allocated bocks of memory. See Attribute and its subclasses for details.

C attribute plug-ins

CShadow’s #shadow_attr methods check for one (no more, no less) matching attribute class. The details of the matching depend on the attribute class. See Attribute for details. Additional attribute classes can easily be added simply by subclassing CShadow::Attribute.

Namespace usage

TODO: prefix every attribute, method, constant name with “cshadow_”?

Using CShadow classes with YAML

CShadow classes can serialize via YAML. Both shadow attributes and ordinary ruby instance variables are serialized. No additional attribute code is needed, because of the generic load/dump mechanism used in cshadow (attributes are pushed into or shifted out of an array, which is passed to or from the serialization protocol, either Marshal or YAML). The ordering of shadow attributes is preserved when dumping to a YAML string.

The only user requirement is that, before attempting to load shadow class instances from YAML string, the shadow class types must be registered with YAML. This is simple:

CShadow.allow_yaml

This method may be called before or after committing. Calling this method also loads the standard yaml library, if it has not already been loaded.

See examples/yaml.rb

Common problems and their solutions

Do you get a NameError because accessor methods are not defined? Make sure you commit your class.

You assign to an attribute but it doesn’t change? Ruby assumes that, in an assignment like “x=1”, the left hand side is a local variable. For all writer methods, whether Ruby or C attributes, you need to do “self.x=1”.

Notes

  • As with most modules, including CShadow more than once has no effect.

However, CShadow cannot currently be included in another module.

  • In addition to the included examples, the RedShift project uses CShadow

extensively. See rubyforge.org/projects/redshift.

Limitations:

  • Hash args are ordered unpredictably, so if struct member order is significant (for example, because you want to pass the struct to C code that expects it that way), use a separate declare statement for each member. Also, take note of the self pointer at the beginning of the struct.

  • Creating a ruby+shadow object has a bit more time/space overhead than just a C object, so CShadow may not be the best mechansism for managing heap allocated data (for example, if you want lots of 2-element arrays). Also, CShadow objects are fixed in size (though their members can point to other structures).

  • CShadow attributes are, of course, not dynamic. They are fixed at the time of #commit. Otherwise, they behave essentially like Ruby attributes, except that they can be accessed only with methods or from C code; they cannot be accessed with the @ notation. Of course, the reader and writer for a shadow attribute can be flagged as protected or private. However, a private writer cannot be used, since by definition private methods can only be called in the receiverless form.

  • CShadow is designed for efficient in-memory structs, not packed, network-ordered data as for example in network protocols. See the bit-struct project for the latter.

To do:

  • It should be easier to get a handle to entities. Below, shadow_attr has been hacked to return a list of pairs of functions. But it should be easier and more general.

  • Optimization: if class A<B, and their free func have the same content, use B’s function in A, and don’t generate a new function for A. Similarly for all the other kinds of functions.

  • Allow

    shadow_attr "int x", "double y"

    or even

    attr_accessor :a, :b, "int x", "double y"

    and (in cgen)

    declare "int x", "double y"

    The ruby name will be extracted from the string using the matching pattern.

  • Generate documentation for the shadow class.

  • Change name to CStruct? CStructure?

  • Find a way to propagate append_features so that CShadow can be included in modules and modules can contribute shadow_attrs.

  • option to omit the “self” pointer, or put it at the end of the struct automatically omit it in a class if no ShadowObjectAttributes point to it?

  • shadow_struct_constructor class method to use DATA_WRAP_STRUCT

  • Use CNativeAttribute as a default attribute? Or use the attr class hierarchy to supply defaults?

Defined Under Namespace

Modules: CShadowClassMethods Classes: AttrClassMethod, AttrCodeAccumulator, AttrMethod, Attribute, BooleanAttribute, CNativeAttribute, CharPointerAttribute, DoubleAttribute, DoublePointerAttribute, FreeFunction, IntAttribute, Library, LongAttribute, MarkFunction, ObjectAttribute, PointerAttribute, ShadowObjectAttribute, ShortAttribute

Constant Summary collapse

SHADOW_SUFFIX = 
"_Shadow"
AttributeTypes = 
[]

Class Method Summary collapse

Instance Method Summary collapse

Class Method Details

.append_features(base_class) ⇒ Object

:nodoc:



977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
 
# File 'lib/cgen/cshadow.rb', line 977

def self.append_features base_class # :nodoc:
  unless base_class.is_a? Class
    raise TypeError, "CShadow can be included only in a Class"
  end

  unless base_class.ancestors.include? self
    base_class.class_eval {@base_class = self; @persistent = true}
    base_class.extend CShadowClassMethods

    class << base_class
      ## why can't these be in CShadowClassMethods?

      alias really_protected protected
      def protected(*args)
        (@to_be_protected ||= []).concat args
      end

      alias really_private private
      def private(*args)
        (@to_be_private ||= []).concat args
      end

      def protect_shadow_attrs
        if defined?(@to_be_protected) and @to_be_protected
          really_protected(*@to_be_protected)
        end
        if defined?(@to_be_private) and @to_be_private
          really_private(*@to_be_private)
        end
        ## should undo the aliasing
      end
      public :protect_shadow_attrs

    end

  end

  super
end

Instance Method Details

#each_attr_value ⇒ Object

Iterate over each shadow attr and instance var of self, yielding the attr name and value in this instance to the block. Differs in three ways from CShadow.each_shadow_attr: it is an instance method of shadow objects, it iterates over both shadow attrs and instance vars, and it yields both the name and the value. (In the case of instance vars, the name does not include the “@”.)



1069
1070
1071
1072
1073
1074
1075
1076
1077
 
# File 'lib/cgen/cshadow.rb', line 1069

def each_attr_value # :yields: attr_name, attr_value
  values = _dump_data
  self.class.shadow_attrs.each_with_index do |attr, i|
    yield attr.var.to_s, values[i]
  end
  instance_variables.each do |ivar|
    yield ivar[1..-1], instance_variable_get(ivar)
  end
end

#each_persistent_attr ⇒ Object

:yields: attr_name



1091
1092
1093
1094
1095
1096
1097
1098
1099
 
# File 'lib/cgen/cshadow.rb', line 1091

def each_persistent_attr # :yields: attr_name
  psa = self.class.shadow_attrs.select {|attr| attr.persists}
  psa.each do |attr|
    yield attr.var.to_s
  end
  instance_variables.each do |ivar|
    yield ivar[1..-1]
  end
end

#each_persistent_attr_value ⇒ Object

Like #each_attr_value, but limited to attr declared as persistent.



1080
1081
1082
1083
1084
1085
1086
1087
1088
1089
 
# File 'lib/cgen/cshadow.rb', line 1080

def each_persistent_attr_value # :yields: attr_name, attr_value
  values = _dump_data
  psa = self.class.shadow_attrs.select {|attr| attr.persists}
  psa.each_with_index do |attr, i|
    yield attr.var.to_s, values[i]
  end
  instance_variables.each do |ivar|
    yield ivar[1..-1], instance_variable_get(ivar)
  end
end

#encode_with(coder) ⇒ Object 



1101
1102
1103
1104
1105
 
# File 'lib/cgen/cshadow.rb', line 1101

def encode_with coder
  each_persistent_attr_value do |attr, value|
    coder[attr] = value
  end
end

#init_with(coder) ⇒ Object 



1107
1108
1109
1110
1111
1112
1113
1114
1115
1116
1117
 
# File 'lib/cgen/cshadow.rb', line 1107

def init_with coder
  psa = self.class.shadow_attrs.select {|attr| attr.persists}
  psvars = psa.map{|attr|attr.var.to_s}

  from_array = psvars.map {|sv| coder[sv]}
  _load_data(from_array)
  
  (coder.map.keys - psvars).each do |ivar|
    obj.instance_variable_set("@#{ivar}", coder[ivar])
  end
end

#inspect ⇒ Object

:nodoc:



1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
 
# File 'lib/cgen/cshadow.rb', line 1049

def inspect # :nodoc:
  attrs = []
  seen = {self => true}
  each_attr_value do |attr, value|
    if seen[value]
      attrs << "#{attr}=#{value}"
    else
      attrs << "#{attr}=#{value.inspect}"
    end
    seen[value] = true
  end
  super.sub(/(?=>\z)/, " " + attrs.join(", "))
end