Module: Smash::CloudPowers::Storage

Includes:
AwsResources
Included in:
Smash::CloudPowers, Delegator
Defined in:
lib/cloud_powers/storage.rb

Instance Method Summary collapse

Methods included from AwsResources

#ec2, #image, #kinesis, #region, #s3, #sns, #sqs

Methods included from Zenv

#env_vars, #file_tree_search, #i_vars, #project_root, #project_root=, #system_vars, #zfind

Methods included from Helper

#attr_map!, #available_resources, #called_from, #create_logger, #deep_modify_keys_with, #format_error_message, #log_file, #logger, #modify_keys_with, #smart_retry, #task_path, #task_require_path, #to_camel, #to_hyph, #to_i_var, #to_pascal, #to_ruby_file_name, #to_snake, #update_message_body, #valid_json?, #valid_url?

Methods included from Auth

creds, region

Instance Method Details

#search(bucket, pattern) ⇒ Object

Search through a bucket to find a file, based on a regex

Parameters

  • bucket String - the bucket to search through in AWS
  • pattern Regex - the Regex pattern you want to use for the search

Example matches = search('neuronTasks', /[Dd]emo\w*/)

=> ['Aws::S3::Type::ListObjectOutPut','Aws::S3::Type::ListObjectOutPut',...] # anything that matched that regex

matches.first.contents.size

=> 238934 # integer representation of the file size



66
67
68
69
70
 
# File 'lib/cloud_powers/storage.rb', line 66

def search(bucket, pattern)
  s3.list_objects(bucket: bucket).contents.select do |o|
    o.key =~ pattern
  end
end

#send_logs_to_s3 ⇒ Object

Send the log files to the S3 log file bucket

Returns Aws::S3::Type::PutObjectOutput



76
77
78
79
80
81
82
83
84
 
# File 'lib/cloud_powers/storage.rb', line 76

def send_logs_to_s3
  File.open(log_file) do |file|
    s3.put_object(
      bucket: log_bucket,
      key: instance_id,
      body: file
    )
  end
end

#source_task(file) ⇒ Object

Searches a local task storage location for the given file name if it exists - exit the method if it does not exist - get the file from s3 and place it in the directory that was just searched bucket using #zfind()

Parameters

  • file String - the name of the file we're searching for

Returns nothing

Example

  • file tree project_root- | |_sub_directory | |_current_file.rb |_task_storage | |_demorific.rb | |_foobar.rb
  • code: 
    source_task('demorific')
# file tree doesn't change because this file exists
source_task('custom_greetings')
# file tree now looks like this
  • new file tree project_root- | |_sub_directory | |_current_file.rb |_task_storage | |_demorific.rb | |_foobar.rb | |_custom_greetings.js # could be an after effects JS script


41
42
43
44
45
46
47
48
49
50
51
52
 
# File 'lib/cloud_powers/storage.rb', line 41

def source_task(file)
  # TODO: better path management
  bucket = zfind('task storage')
  unless task_path(file).exist?
    objects = s3.list_objects(bucket: bucket).contents.select do |f|
      /#{Regexp.escape file}/i =~ f.key
    end
    objects.each do |obj|
      s3.get_object(bucket: bucket, key: obj.key, response_target: task_path(file))
    end
  end
end