MagicLoader

What if you could just point something at a big directory full of code and have everything just automagically load regardless of the dependency structure?

Wouldn't that be nice? Well, now you can!

require 'magic_loader'

MagicLoader uses a special algorithm, sort of like Bundler, to calculate the order in which various files in your project must be required in order for your project to load successfully. If you've been manually maintaining a dependencies list, MagicLoader provides for your code the same automation that Bundler provides for your gem dependencies.

The "magicload" Rake task

The main way MagicLoader should be used is as a Rake task. Inside of your project's Rakefile, do:


 require 'magicloader/tasks'
 
 MagicLoader::Task.new 'lib/my_project', 
  :target => 'lib/my_project.rb',
  :strip  => 'lib/', 
  :name   => 'magicload'

What does this do exactly? Let's dig in.

The MagicLoader::Task.new method takes a list of strings as its arguments, with an option options hash on the end (of course options are optional!)

In this example, we have only one string, which is the path to our project. However, you can have multiple strings, and they don't need to just be paths, they can be globs, too! For more information on the string arguments, see "The wonderful require_all" method below.

Pay special attention to the :target option. If you leave this out, MagicLoader will print what it would've generated to standard output. However, if you set the target, you can annotate a particular file with a MagicLoader Magic Block. What's that you ask? Well it looks a bit like this:


  #-----BEGIN MAGICLOADER MAGIC BLOCK-----
  # Automagically generated by MagicLoader. Editing may
  # result in bad juju. Edit at your own risk!
  require "my_project/foo.rb"
  require "my_project/bar.rb"
  require "my_project/baz.rb"
  require "my_project/qux.rb"
  #------END MAGICLOADER MAGIC BLOCK------

Here MagicLoader is automagically writing the dependency order of your library into the master file for your project. The dependency order is appended to the end of the file if it doesn't exist or updated if it does. The rest of your file remains unchanged.

MagicLoader loads up your whole project within the Rake task, which is a nice, clean-room, thread-free environment where your code can be safely loaded without crazy wonky stuff going on. Here MagicLoader uses its magic algorithm to calculate the dependencies, then writes them all out for you in the MagicLoader Magic Block.

Now, let's dig into the rest of the options hash. You might be able to guess what the options do already, but if you can't, here's some help:

  • target: output file to annotate. See above.

  • strip: remove the given leading string from all of the output paths written to the target file. Optionally, you can give an arbitrary regular expression, and whatever it matches will be removed from the string. This is good if you have a leading directory prefix you want to remove from the generated output files.

  • name: the name of the Rake task to define. It defaults to "magicload" if you don't specify it yourself, so specifying it explicitly as "magicload" is a bit redundant. Otherwise, you can set it to whatever you want.

  • clean: true or false boolean. The "clean" option is especially useful if your target file is included in the Rake environment. If this happens, it will distrupt MagicLoader's typical operation. Setting :clean => true will remove previous MagicLoader directives from the target file before attempting to regenerate them, affording you a pristine environment in which to compute your code dependencies.

The wonderful require_all method

You can probably guess what MagicLoader.require_all does: it requires a whole bunch of files. It's highly configurable and can pull in globs or whole directories of code.

However, there are some caveats about the require_all method: it's slow and it isn't thread safe. That said, if you would like runtime code dependency management, then proceed, young sorcerer.

The easiest way to use require_all is to just point it at a directory containing a bunch of .rb files:

MagicLoader.require_all 'lib'

This will find all the .rb files under the lib directory (including all subdirectories as well) and load them.

The proper order to in which to load them is determined automatically. If the dependencies between the matched files are unresolvable, it will throw the first unresolvable NameError.

You can also give it a glob, which will enumerate all the matching files:

MagicLoader.require_all 'lib/*/.rb'

It will also accept an array of files:

MagicLoader.require_all Dir.glob("blah/*/.rb").reject { |f| stupid_file? f }

Or if you want, just list the files directly as arguments:

MagicLoader.require_all 'lib/a.rb', 'lib/b.rb', 'lib/c.rb', 'lib/d.rb'

So what's the magic?

MagicLoader is just a Ruby library, after all. There's no magic but what you don't understand. I didn't invent the approach this gem uses. It was shamelessly stolen from Merb (which apparently stole it from elsewhere).

Here's how it works:

  • Enumerate the files to be loaded
  • Try to load all of the files. If we encounter a NameError loading a particular file, store that file in a "try to load it later" list.
  • If all the files loaded, great, we're done! If not, go through the "try to load it later" list again rescuing NameErrors the same way.
  • If we walk the whole "try to load it later" list and it doesn't shrink at all, we've encountered an unresolvable dependency. In this case, require_all will rethrow the first NameError it encountered.

Questions? Comments? Concerns?

You can reach the author on github or freenode: "tarcieri"

or on Twitter: @bascule or email: [email protected]

Got issues with require_all to report? Post 'em here:

Github Tracker

License

MIT (see the LICENSE file for details)