Coque

Create, manage, and interop with shell pipelines from Ruby. Like Plumbum, for Ruby, with native (Ruby) code streaming integration.

Installation

Add to your gemfile:

gem 'coque'

Usage

Create Coque commands:

cmd = Coque["echo", "hi"]
# => <Coque::Sh ["echo", "hi"]>

And run them:

res = cmd.run
# => #<Coque::Result:0x007feb5930e408 @out=#<IO:fd 13>, @pid=58688>
res.to_a
# => ["hi"]

Or pipe them:

pipeline = cmd | Coque["wc", "-c"]
# => #<Coque::Pipeline:0x007feb598730b0 @commands=[<Coque::Sh ["echo", "hi"]>, <Coque::Sh ["wc", "-c"]>]>
pipeline.run.to_a
# => ["3"]

Coque can also create "Rb" commands, which integrate Ruby code with streaming, line-wise processing of other commands:

c1 = Coque["printf", '"a\nb\nc\n"']
c2 = Coque.rb { |line| puts line.upcase }
(c1 | c2).run.to_a
# => ["A", "B", "C"]

Rb commands can also take "pre" and "post" blocks

dict = Coque["cat", "/usr/share/dict/words"]
rb_wc = Coque.rb { @lines += 1 }.pre { @lines = 0 }.post { puts @lines }

(dict | rb_wc).run.to_a
# => ["235886"]

Commands can have Stdin, Stdout, and Stderr redirected

(Coque["echo", "hi"] > "/tmp/hi.txt").run.wait
File.read("/tmp/hi.txt")
# => "hi\n"

(Coque["head", "-n", "4"] < "/usr/share/dict/words").run.to_a
# => ["A", "a", "aa", "aal"]

(Coque["cat", "/doesntexist.txt"] >= "/tmp/error.txt").run.wait
File.read("/tmp/error.txt")
# => "cat: /doesntexist.txt: No such file or directory\n"

Coque commands can also be derived from a Coque::Context, which enables changing directory, setting environment variables, and unsetting child env:

c = Coque.context
c["pwd"].run.to_a
# => ["/Users/worace/code/coque"]

Coque.context.chdir("/tmp")["pwd"].run.to_a
# => ["/private/tmp"]

Coque.context.setenv("my_key": "pizza")["echo", "$my_key"].run.to_a
# => ["pizza"]

ENV["my_key"] = "pizza"
Coque["echo", "$my_key"].run.to_a
# => ["pizza"]

Coque.context.disinherit_env["echo", "$my_key"].to_a
# => [""]

Coque also includes a Coque.source helper for feeding Ruby enumerables into shell pipelines:

(Coque.source(1..500) | Coque["wc", "-l"]).run.to_a
# => ["500"]

Streaming Performance

Should be little overhead compared with the equivalent pipeline from a standard shell.

From zsh:

head -c 100000000 /dev/urandom | pv | wc -c
95.4MiB 0:00:06 [14.1MiB/s] [      <=>      ]
 100000000

With coque:

p = Coque["head", "-c", "100000000", "/dev/urandom"] | Coque["pv"] | Coque["wc", "-c"]
p.run.wait
95.4MiB 0:00:06 [14.6MiB/s] [           <=> ]

Development

• Setup local environment with standard bundle
• Run tests with rake
• See code coverage output in coverage/
• Start a pry console with bin/console
• Install current dev version with rake install
• Use rake release to release after bumping lib/coque/version.rb
• New issues welcome

Further Reading / Prior Art

The concept and API for this library was heavily inspired by Python's excellent Plumbum library.

I relied on many resources to understand Ruby's great facilities for Process creation and manipulation. Some highlights include:

• Avdi Grimm's A dozen (or so) ways to start sub-processes in Ruby (Part 1, Part 2, Part 3)
• Ryan Tomayko's I like Unicorn because it's Unix
• Jesse Storimer's Working With Unix Processes
• Brandon Wamboldt's blog series: How bash redirection works, How Linux pipes work, and Understanding how Linux creates processes

License

The gem is available as open source under the terms of the MIT License.

Code of Conduct

Everyone interacting in the Coque project’s codebases, issue trackers, chat rooms and mailing lists is expected to follow the code of conduct.