Class: Jets::Builders::CodeBuilder
- Inherits:
-
Object
- Object
- Jets::Builders::CodeBuilder
- Extended by:
- Memoist
- Includes:
- AwsServices, Util
- Defined in:
- lib/jets/builders/code_builder.rb
Instance Attribute Summary collapse
-
#full_project_path ⇒ Object
readonly
Returns the value of attribute full_project_path.
Class Method Summary collapse
-
.tmp_code ⇒ Object
Group all the path settings together here.
Instance Method Summary collapse
- #build ⇒ Object
- #build_lambda_layer ⇒ Object
- #cache_check_message ⇒ Object
-
#calculate_md5s ⇒ Object
Resolves the chicken-and-egg problem with md5 checksums.
- #check_agree ⇒ Object
- #check_code_size! ⇒ Object
- #check_ruby_version ⇒ Object
-
#clean_start ⇒ Object
Cleans out non-cached files like code-*.zip in Jets.build_root for a clean start.
- #code_finish ⇒ Object
- #code_setup ⇒ Object
-
#compile_assets ⇒ Object
This happens in the current app directory not the tmp code for simplicity.
-
#compile_rails_assets ⇒ Object
This happens in the current app directory not the tmp code for simplicity This is because the node likely been set up correctly there.
-
#copy_internal_jets_code ⇒ Object
Materialized internal code into actually user Jets app as part of the deploy process.
-
#copy_project ⇒ Object
Copy project into temporary directory.
- #create_zip_files ⇒ Object
-
#dir_size(folder) ⇒ Object
Thanks stackoverflow.com/questions/9354595/recursively-getting-the-size-of-a-directory Seems to overestimate a little bit but close enough.
- #disable_webpacker_middleware ⇒ Object
- #exist_on_s3?(filename) ⇒ Boolean
- #generate_shims ⇒ Object
-
#initialize ⇒ CodeBuilder
constructor
A new instance of CodeBuilder.
-
#move_node_modules(source_folder, dest_folder) ⇒ Object
Move the node modules to the tmp build folder to speed up project copying.
- #package_ruby ⇒ Object
- #rack_packager ⇒ Object
-
#rails? ⇒ Boolean
Rudimentary rails detection Duplicated in builders/reconfigure_rails.rb.
-
#rails_api? ⇒ Boolean
Rudimentary rails api detection Duplicated in builders/reconfigure_rails.rb Another way of checking is loading a rails console and checking Rails.application.config.api_only Using this way for simplicity.
- #rails_assets(cmd, jets_root:) ⇒ Object
-
#reconfigure_development_webpacker ⇒ Object
Bit hacky but this saves the user from accidentally forgetting to change this when they deploy a jets project in development mode.
-
#reconfigure_rails ⇒ Object
TODO: Move logic into plugin instead.
- #ruby_packager ⇒ Object
- #ruby_version_supported? ⇒ Boolean
- #s3_base_url ⇒ Object
- #s3_bucket ⇒ Object
-
#store_s3_base_url ⇒ Object
Store s3 base url is needed for asset serving from s3 later.
- #tmp_code ⇒ Object
- #webpacker_included? ⇒ Boolean
- #write_s3_base_url(full_path) ⇒ Object
Methods included from AwsServices
#apigateway, #aws_lambda, #cfn, #dynamodb, #logs, #s3, #s3_resource, #sns, #sqs, #sts
Methods included from AwsServices::StackStatus
#lookup, #stack_exists?, #stack_in_progress?
Methods included from AwsServices::GlobalMemoist
Constructor Details
#initialize ⇒ CodeBuilder
Returns a new instance of CodeBuilder.
21 22 23 24 25 26 |
# File 'lib/jets/builders/code_builder.rb', line 21 def initialize # Expanding to the full path and capture now. # Dir.chdir gets called later and we'll lose this info. @full_project_path = File.(Jets.root) + "/" @version_purger = Purger.new end |
Instance Attribute Details
#full_project_path ⇒ Object (readonly)
Returns the value of attribute full_project_path.
20 21 22 |
# File 'lib/jets/builders/code_builder.rb', line 20 def full_project_path @full_project_path end |
Class Method Details
Instance Method Details
#build ⇒ Object
28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 |
# File 'lib/jets/builders/code_builder.rb', line 28 def build check_ruby_version @version_purger.purge clean_start compile_assets # easier to do before we copy the project because node and yarn has been likely setup in the that dir compile_rails_assets copy_project Dir.chdir("#{stage_area}/code") do # These commands run from project root code_setup package_ruby code_finish end end |
#build_lambda_layer ⇒ Object
348 349 350 351 352 |
# File 'lib/jets/builders/code_builder.rb', line 348 def build_lambda_layer return if Jets.poly_only? lambda_layer = LambdaLayer.new lambda_layer.build end |
#cache_check_message ⇒ Object
359 360 361 362 363 |
# File 'lib/jets/builders/code_builder.rb', line 359 def if File.exist?("#{Jets.build_root}/cache") puts "The #{Jets.build_root}/cache folder exists. Incrementally re-building the jets using the cache. To clear the cache: rm -rf #{Jets.build_root}/cache" end end |
#calculate_md5s ⇒ Object
Resolves the chicken-and-egg problem with md5 checksums. The handlers need to reference files with the md5 checksum. The files are the:
jets/code/rack-checksum.zip
jets/code/opt-checksum.zip
We compute the checksums before we generate the node shim handlers.
52 53 54 |
# File 'lib/jets/builders/code_builder.rb', line 52 def calculate_md5s Md5.compute! # populates Md5.checksums hash end |
#check_agree ⇒ Object
343 344 345 346 |
# File 'lib/jets/builders/code_builder.rb', line 343 def check_agree agree = Jets::Gems::Agree.new agree.prompt end |
#check_code_size! ⇒ Object
105 106 107 |
# File 'lib/jets/builders/code_builder.rb', line 105 def check_code_size! CodeSize.check! end |
#check_ruby_version ⇒ Object
365 366 367 368 369 370 371 |
# File 'lib/jets/builders/code_builder.rb', line 365 def check_ruby_version unless ruby_version_supported? puts "You are using Ruby version #{RUBY_VERSION} which is not supported by Jets." ruby_variant = Jets::RUBY_VERSION.split('.')[0..1].join('.') + '.x' abort("Jets uses Ruby #{Jets::RUBY_VERSION}. You should use a variant of Ruby #{ruby_variant}".color(:red)) end end |
#clean_start ⇒ Object
Cleans out non-cached files like code-*.zip in Jets.build_root for a clean start. Also ensure that the /tmp/jets/project build root exists.
Most files are kept around after the build process for inspection and debugging. So we have to clean out the files. But we only want to clean out some of the files.
270 271 272 273 |
# File 'lib/jets/builders/code_builder.rb', line 270 def clean_start Dir.glob("#{Jets.build_root}/code/code-*.zip").each { |f| FileUtils.rm_f(f) } FileUtils.mkdir_p(Jets.build_root) # /tmp/jets/demo end |
#code_finish ⇒ Object
92 93 94 95 96 97 98 99 100 101 102 103 |
# File 'lib/jets/builders/code_builder.rb', line 92 def code_finish # Reconfigure code store_s3_base_url disable_webpacker_middleware copy_internal_jets_code # Code prep and zipping check_code_size! calculate_md5s # must be called before generate_shims and create_zip_files generate_shims create_zip_files end |
#code_setup ⇒ Object
88 89 90 |
# File 'lib/jets/builders/code_builder.rb', line 88 def code_setup reconfigure_development_webpacker end |
#compile_assets ⇒ Object
This happens in the current app directory not the tmp code for simplicity. This is because the node and yarn has likely been set up correctly there.
187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 |
# File 'lib/jets/builders/code_builder.rb', line 187 def compile_assets if ENV['JETS_SKIP_ASSETS'] puts "Skip compiling assets".color(:yellow) # useful for debugging return end headline "Compling assets in current project directory" return unless webpacker_included? sh("yarn install") webpack_command = File.exist?("#{Jets.root}/bin/webpack") ? "bin/webpack" : `which webpack`.strip sh "JETS_ENV=#{Jets.env} #{webpack_command}" end |
#compile_rails_assets ⇒ Object
This happens in the current app directory not the tmp code for simplicity This is because the node likely been set up correctly there.
217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 |
# File 'lib/jets/builders/code_builder.rb', line 217 def compile_rails_assets return unless Jets.rack? && rails? && !rails_api? if ENV['JETS_SKIP_ASSETS'] puts "Skip compiling rack assets".color(:yellow) # useful for debugging return end # Need to capture JETS_ROOT since can be changed by Turbo mode jets_root = Jets.root Bundler.with_clean_env do # Switch gemfile for Afterburner mode gemfile = ENV['BUNDLE_GEMFILE'] ENV['BUNDLE_GEMFILE'] = "#{jets_root}/rack/Gemfile" sh "cd #{jets_root} && bundle install" ENV['BUNDLE_GEMFILE'] = gemfile rails_assets(:clobber, jets_root: jets_root) rails_assets(:precompile, jets_root: jets_root) end end |
#copy_internal_jets_code ⇒ Object
Materialized internal code into actually user Jets app as part of the deploy process. Examples of things that we might materialize:
Views
Simple Functions
For functions, We copy the files into the project because we cannot require simple functions directly since they are wrapped by an anonymous class.
117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 |
# File 'lib/jets/builders/code_builder.rb', line 117 def copy_internal_jets_code files = [] mailers_controller = Jets::Router.has_controller?("Jets::MailersController") if mailers_controller files << "app/controllers/jets/mailers_controller.rb" files << "app/views/jets/mailers" files << "app/helpers/jets/mailers_helper.rb" end files.each do |relative_path| src = File.("../internal/#{relative_path}", File.dirname(__FILE__)) dest = "#{"#{stage_area}/code"}/#{relative_path}" FileUtils.mkdir_p(File.dirname(dest)) FileUtils.cp_r(src, dest) end end |
#copy_project ⇒ Object
Copy project into temporary directory. Do this so we can keep the project directory untouched and we can also remove a bunch of unnecessary files like logs before zipping it up.
278 279 280 281 282 283 284 285 286 287 288 289 290 |
# File 'lib/jets/builders/code_builder.rb', line 278 def copy_project headline "Copying current project directory to temporary build area: #{"#{stage_area}/code"}" FileUtils.rm_rf("#{build_area}/stage") # clear out from previous build's stage area FileUtils.mkdir_p("#{build_area}/stage") FileUtils.rm_rf("#{stage_area}/code") # remove current code folder move_node_modules(Jets.root, Jets.build_root) begin # puts "cp -r #{@full_project_path} #{"#{stage_area}/code"}".color(:yellow) # uncomment to debug Jets::Util.cp_r(@full_project_path, "#{stage_area}/code") ensure move_node_modules(Jets.build_root, Jets.root) # move node_modules directory back end end |
#create_zip_files ⇒ Object
64 65 66 67 68 69 70 71 72 73 74 75 76 |
# File 'lib/jets/builders/code_builder.rb', line 64 def create_zip_files folders = Md5.stage_folders # Md5.stage_folders ["stage/bundled", "stage/code"] folders.each do |folder| zip = Md5Zip.new(folder) if exist_on_s3?(zip.md5_name) puts "Already exists: s3://#{s3_bucket}/jets/code/#{zip.md5_name}" else zip = Md5Zip.new(folder) zip.create end end end |
#dir_size(folder) ⇒ Object
Thanks stackoverflow.com/questions/9354595/recursively-getting-the-size-of-a-directory Seems to overestimate a little bit but close enough.
137 138 139 140 141 142 |
# File 'lib/jets/builders/code_builder.rb', line 137 def dir_size(folder) Dir.glob(File.join(folder, '**', '*')) .select { |f| File.file?(f) } .map{ |f| File.size(f) } .inject(:+) end |
#disable_webpacker_middleware ⇒ Object
179 180 181 182 183 |
# File 'lib/jets/builders/code_builder.rb', line 179 def disable_webpacker_middleware full_path = "#{"#{stage_area}/code"}/config/disable-webpacker-middleware.txt" FileUtils.mkdir_p(File.dirname(full_path)) FileUtils.touch(full_path) end |
#exist_on_s3?(filename) ⇒ Boolean
78 79 80 81 82 83 84 85 86 |
# File 'lib/jets/builders/code_builder.rb', line 78 def exist_on_s3?(filename) s3_key = "jets/code/#{filename}" begin s3.head_object(bucket: s3_bucket, key: s3_key) true rescue Aws::S3::Errors::NotFound false end end |
#generate_shims ⇒ Object
56 57 58 59 60 61 62 |
# File 'lib/jets/builders/code_builder.rb', line 56 def generate_shims headline "Generating shims in the handlers folder." # Crucial that the Dir.pwd is in the tmp_code because for # Jets::Builders::app_files because Jets.boot set ups # autoload_paths and this is how project classes are loaded. Jets::Builders::HandlerGenerator.build! end |
#move_node_modules(source_folder, dest_folder) ⇒ Object
Move the node modules to the tmp build folder to speed up project copying. A little bit risky because a ctrl-c in the middle of the project copying results in a missing node_modules but user can easily rebuild that.
Tesing shows 6.623413 vs 0.027754 speed improvement.
297 298 299 300 301 302 303 |
# File 'lib/jets/builders/code_builder.rb', line 297 def move_node_modules(source_folder, dest_folder) source = "#{source_folder}/node_modules" dest = "#{dest_folder}/node_modules" if File.exist?(source) FileUtils.mv(source, dest) end end |
#package_ruby ⇒ Object
330 331 332 333 334 335 336 337 338 339 340 341 |
# File 'lib/jets/builders/code_builder.rb', line 330 def package_ruby return if Jets.poly_only? check_agree ruby_packager.install reconfigure_rails # call here after "#{stage_area}/code" is available rack_packager.install ruby_packager.finish # by this time we have a /tmp/jets/demo/stage/code/vendor/gems rack_packager.finish build_lambda_layer end |
#rack_packager ⇒ Object
325 326 327 |
# File 'lib/jets/builders/code_builder.rb', line 325 def rack_packager RackPackager.new("#{tmp_code}/rack") end |
#rails? ⇒ Boolean
Rudimentary rails detection Duplicated in builders/reconfigure_rails.rb
248 249 250 251 252 |
# File 'lib/jets/builders/code_builder.rb', line 248 def rails? config_ru = "#{Jets.root}/rack/config.ru" return false unless File.exist?(config_ru) !IO.readlines(config_ru).grep(/Rails.application/).empty? end |
#rails_api? ⇒ Boolean
Rudimentary rails api detection Duplicated in builders/reconfigure_rails.rb Another way of checking is loading a rails console and checking Rails.application.config.api_only Using this way for simplicity.
258 259 260 261 262 |
# File 'lib/jets/builders/code_builder.rb', line 258 def rails_api? config_app = "#{Jets.root}/rack/config/application.rb" return false unless File.exist?(config_app) !IO.readlines(config_app).grep(/config.api_only.*=.*true/).empty? end |
#rails_assets(cmd, jets_root:) ⇒ Object
239 240 241 242 243 244 |
# File 'lib/jets/builders/code_builder.rb', line 239 def rails_assets(cmd, jets_root:) # rake is available in both rails 4 and 5. rails command only in 5 command = "bundle exec rake assets:#{cmd} --trace" command = "RAILS_ENV=#{Jets.env} #{command}" unless Jets.env.development? sh("cd #{jets_root}/rack && #{command}") end |
#reconfigure_development_webpacker ⇒ Object
Bit hacky but this saves the user from accidentally forgetting to change this when they deploy a jets project in development mode
307 308 309 310 311 312 313 314 315 316 317 318 |
# File 'lib/jets/builders/code_builder.rb', line 307 def reconfigure_development_webpacker return unless Jets.env.development? headline "Reconfiguring webpacker development settings for AWS Lambda." webpacker_yml = "#{"#{stage_area}/code"}/config/webpacker.yml" return unless File.exist?(webpacker_yml) config = YAML.load_file(webpacker_yml) config["development"]["compile"] = false # force this to be false for deployment new_yaml = YAML.dump(config) IO.write(webpacker_yml, new_yaml) end |
#reconfigure_rails ⇒ Object
TODO: Move logic into plugin instead
355 356 357 |
# File 'lib/jets/builders/code_builder.rb', line 355 def reconfigure_rails ReconfigureRails.new("#{"#{stage_area}/code"}/rack").run end |
#ruby_packager ⇒ Object
320 321 322 |
# File 'lib/jets/builders/code_builder.rb', line 320 def ruby_packager RubyPackager.new(tmp_code) end |
#ruby_version_supported? ⇒ Boolean
373 374 375 376 377 378 379 380 381 |
# File 'lib/jets/builders/code_builder.rb', line 373 def ruby_version_supported? pattern = /(\d+)\.(\d+)\.(\d+)/ md = RUBY_VERSION.match(pattern) ruby = {major: md[1], minor: md[2]} md = Jets::RUBY_VERSION.match(pattern) jets = {major: md[1], minor: md[2]} ruby[:major] == jets[:major] && ruby[:minor] == jets[:minor] end |
#s3_base_url ⇒ Object
158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 |
# File 'lib/jets/builders/code_builder.rb', line 158 def s3_base_url # Allow user to set assets.base_url # # Jets.application.configure do # config.assets.base_url = "https://cloudfront.com/my/base/path" # end # return Jets.config.assets.base_url if Jets.config.assets.base_url region = Jets.aws.region asset_base_url = region == 'us-east-1' ? "https://s3.amazonaws.com" : "https://s3-#{region}.amazonaws.com" "#{asset_base_url}/#{s3_bucket}/jets" # s3_base_url end |
#s3_bucket ⇒ Object
175 176 177 |
# File 'lib/jets/builders/code_builder.rb', line 175 def s3_bucket Jets.aws.s3_bucket end |
#store_s3_base_url ⇒ Object
Store s3 base url is needed for asset serving from s3 later. Need to package this as part of the code so we have a reference to it. At this point the minimal stack exists, so we can grab it with the AWS API. We do not want to grab this as part of the live request because it is slow.
148 149 150 151 |
# File 'lib/jets/builders/code_builder.rb', line 148 def store_s3_base_url write_s3_base_url("#{stage_area}/code/config/s3_base_url.txt") write_s3_base_url("#{stage_area}/rack/config/s3_base_url.txt") if Jets.rack? end |
#tmp_code ⇒ Object
388 389 390 |
# File 'lib/jets/builders/code_builder.rb', line 388 def tmp_code self.class.tmp_code end |
#webpacker_included? ⇒ Boolean
203 204 205 206 207 208 209 210 211 212 213 |
# File 'lib/jets/builders/code_builder.rb', line 203 def webpacker_included? # Old code, leaving around for now: # Thanks: https://stackoverflow.com/questions/4195735/get-list-of-gems-being-used-by-a-bundler-project # webpacker_loaded = Gem.loaded_specs.keys.include?("webpacker") # return unless webpacker_loaded # Checking this way because when using jets standalone for Afterburner mode we don't want to run into # bundler gem collisions. TODO: figure out the a better way to handle the collisions. lines = IO.readlines("#{Jets.root}/Gemfile") lines.detect { |l| l =~ /webpacker/ } end |
#write_s3_base_url(full_path) ⇒ Object
153 154 155 156 |
# File 'lib/jets/builders/code_builder.rb', line 153 def write_s3_base_url(full_path) FileUtils.mkdir_p(File.dirname(full_path)) IO.write(full_path, s3_base_url) end |