Class: Jets::Builders::GemReplacer

Inherits:
Object
  • Object
show all
Extended by:
Memoist
Defined in:
lib/jets/builders/gem_replacer.rb

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(ruby_version, options) ⇒ GemReplacer

Returns a new instance of GemReplacer.



12
13
14
15
16
 
# File 'lib/jets/builders/gem_replacer.rb', line 12

def initialize(ruby_version, options)
  @ruby_version = ruby_version
  @options = options
  @missing_gems = [] # keeps track of gems that are not found in any of the lambdagems sources
end

Instance Attribute Details

#missing_gems ⇒ Object (readonly)

Returns the value of attribute missing_gems.



11
12
13
 
# File 'lib/jets/builders/gem_replacer.rb', line 11

def missing_gems
  @missing_gems
end

Instance Method Details

#compiled_gem_paths ⇒ Object

Use pre-compiled gem because the gem could have development header shared object file dependencies. The shared dependencies are packaged up as part of the pre-compiled gem so it is available in the Lambda execution environment.

Example paths: Macosx:

bundled/gems/ruby/2.5.0/extensions/x86_64-darwin-16/2.5.0-static/nokogiri-1.8.1
bundled/gems/ruby/2.5.0/extensions/x86_64-darwin-16/2.5.0-static/byebug-9.1.0

Official AWS Lambda Linux AMI:

bundled/gems/ruby/2.5.0/extensions/x86_64-linux/2.5.0-static/nokogiri-1.8.1

Circleci Ubuntu based Linux:

bundled/gems/ruby/2.5.0/extensions/x86_64-linux/2.5.0/pg-0.21.0



144
145
146
 
# File 'lib/jets/builders/gem_replacer.rb', line 144

def compiled_gem_paths
  Dir.glob("#{Jets.build_root}/cache/bundled/gems/ruby/*/extensions/**/**/*.{so,bundle}")
end

#compiled_gems ⇒ Object

If there are subfolders compiled_gem_paths might have files deeper in the directory tree. So lets grab the gem name and figure out the unique paths of the compiled gems from there.



127
128
129
 
# File 'lib/jets/builders/gem_replacer.rb', line 127

def compiled_gems
  compiled_gem_paths.map { |p| gem_name_from_path(p) }.uniq# + ["whatever-0.0.1"]
end

#gem_name_from_path(path) ⇒ Object

Input: bundled/gems/ruby/2.5.0/extensions/x86_64-darwin-16/2.5.0-static/byebug-9.1.0 Output: byebug-9.1.0



150
151
152
153
 
# File 'lib/jets/builders/gem_replacer.rb', line 150

def gem_name_from_path(path)
  regexp = /gems\/ruby\/\d+\.\d+\.\d+\/extensions\/.*?\/.*?\/(.*?)\//
  gem_name = path.match(regexp)[1]
end

#missing_gems_message ⇒ Object 



64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
 
# File 'lib/jets/builders/gem_replacer.rb', line 64

def missing_gems_message
  template = <<-EOL
Your project requires compiled gems were not available in any of your lambdagems sources.  Unavailable pre-compiled gems:
<% missing_gems.each do |gem| %>
* <%= gem -%>
<% end %>

Your current lambdagems sources:
<% Jets.config.lambdagems.sources.map do |source| %>
* <%= source -%>
<% end %>

Jets is unable to build a deployment package that will work on AWS Lambda without the required pre-compiled gems. To remedy this, you can:

* Build the gem yourself and add it to your own custom lambdagems sources. Refer to the Lambda Gems Docs: http://rubyonjets.com/docs/lambdagems
* Wait until it added to lambdagems.com. No need to report this to us, as we've already been notified.
* Use another gem that does not require compilation.

Compiled gems usually take some time to figure out how to build as they each depend on different libraries and packages. We make an effort add new gems as soon as we can.
EOL
  erb = ERB.new(template, nil, '-') # trim mode https://stackoverflow.com/questions/4632879/erb-template-removing-the-trailing-line
  erb.result(binding)
end

#run ⇒ Object 



18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
 
# File 'lib/jets/builders/gem_replacer.rb', line 18

def run
  # Checks whether the gem is found on at least one of the lambdagems sources.
  # By the time the loop finishes, found_gems will hold a map of gem names to found
  # url sources. Example:
  #
  #   found_gems = {
  #     "nokogiri-1.8.4" => "https://lambdagems.com",
  #     "pg-0.21.0" => "https://anothersource.com",
  #   }
  #
  found_gems = {}
  compiled_gems.each do |gem_name|
    gem_exists = false
    Jets.config.lambdagems.sources.each do |source|
      exist = Lambdagem::Exist.new(lambdagems_url: source)
      found = exist.check(gem_name)
      # gem exists on at least of the lambdagem sources
      if found
        gem_exists = true
        found_gems[gem_name] = source
        break
      end
    end
    unless gem_exists
      @missing_gems << gem_name
    end
  end

  # Exits early if not all the linux gems are available.
  # It better to error now than deploy a broken package to AWS Lambda.
  # Provide users with message about using their own lambdagems source.
  unless @missing_gems.empty?
    puts missing_gems_message
    exit 1
  end

  # Reaching here means we can download and extract the gems
  Lambdagem.log_level = :info
  found_gems.each do |gem_name, source|
    gem_extractor = Lambdagem::Extract::Gem.new(gem_name, @options.merge(lambdagems_url: source))
    gem_extractor.run
  end

  tidy
end

#tidy ⇒ Object

remove unnecessary files to reduce package size



89
90
91
92
 
# File 'lib/jets/builders/gem_replacer.rb', line 89

def tidy
  tidy_gems("#{@options[:project_root]}/bundled/gems/ruby/*/gems/*")
  tidy_gems("#{@options[:project_root]}/bundled/gems/ruby/*/bundler/gems/*")
end

#tidy_gem(path) ⇒ Object

Clean up some unneeded files to try to keep the package size down In a generated jets app this made a decent 9% difference:

175M test2
191M test3



104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
 
# File 'lib/jets/builders/gem_replacer.rb', line 104

def tidy_gem(path)
  # remove top level tests and cache folders
  Dir.glob("#{path}/*").each do |path|
    next unless File.directory?(path)
    folder = File.basename(path)
    if %w[test tests spec features benchmark cache doc].include?(folder)
      FileUtils.rm_rf(path)
    end
  end

  Dir.glob("#{path}/**/*").each do |path|
    next unless File.file?(path)
    ext = File.extname(path)
    if %w[.rdoc .md .markdown].include?(ext) or
       path =~ /LICENSE|CHANGELOG|README/
      FileUtils.rm_f(path)
    end
  end
end

#tidy_gems(gems_path) ⇒ Object 



94
95
96
97
98
 
# File 'lib/jets/builders/gem_replacer.rb', line 94

def tidy_gems(gems_path)
  Dir.glob(gems_path).each do |gem_path|
    tidy_gem(gem_path)
  end
end