Installation
gem install s3stream
Configuration
Log into Amazon Web Services, and lookup your access key ID and secret. aws-portal.amazon.com/gp/aws/developer/account/index.html?action=access-key
Here, I’ll pretend that your access key is “ABCDEFGHIJKLMNOPQ” and your secret is “12345678901234567890abcde”. Alright then…
% echo 'AWSAccessKeyId=ABCDEFGHIJKLMNOPQ' > ~/.aws-credentials
% echo 'AWSSecretKey=12345678901234567890abcde' >> ~/.aws-credentials
% export AWS_CREDENTIAL_FILE=~/.aws-credentials
- OR -
% export AMAZON_ACCESS_KEY_ID='ABCDEFGHIJKLMNOPQ'
% export AMAZON_SECRET_ACCESS_KEY='12345678901234567890abcde'
- OR -
% export AWS_ACCESS_KEY='ABCDEFGHIJKLMNOPQ'
% export AWS_SECRET_KEY='12345678901234567890abcde'
Usage
% s3stream fetch mybucket myfile.dat > /tmp/myfile.dat
% s3stream store mybucket myfile.dat < /tmp/myfile.dat
If you want to try uploading a bunch of data, but don’t have any handy:
% base64 /dev/urandom \
| head -c $((10 * 1024 * 1024)) \
| s3stream store mybucket 10mb.txt
And if you want to get fancy:
% cat \
<(s3stream fetch mybucket log0.log.gz | gunzip) \
<(s3stream fetch mybucket log1.log.gz | gunzip) \
| lzop -9 \
| s3stream store mybucket combined.log.lzo
If you want to stream an upload in ruby:
S3Stream::Upload.to(:s3object => s3object) do |stream|
stream.write("abc")
stream.write("123")
end