Class: ActiveEncode::EngineAdapters::MediaConvertAdapter

Inherits:
Object
  • Object
show all
Defined in:
lib/active_encode/engine_adapters/media_convert_adapter.rb

Overview

An adapter for using [AWS Elemental MediaConvert](aws.amazon.com/mediaconvert/) to encode.

Note: this adapter does not perform input characterization, does not provide technical metadata on inputs.

## Configuration

ActiveEncode::Base.engine_adapter = :media_convert

ActiveEncode::Base.engine_adapter.role = 'arn:aws:iam::123456789012:role/service-role/MediaConvert_Default_Role'
ActiveEncode::Base.engine_adapter.output_bucket = 'output-bucket'

# optionally and probably not needed

ActiveEncode::Base.engine_adapter.queue = my_mediaconvert_queue_name
ActiveEncode::Base.engine_adapter.log_group = my_log_group_name

## Capturing output information

[AWS Elemental MediaConvert](aws.amazon.com/mediaconvert/) doesn’t provide detailed output information in the job description that can be pulled directly from the service. Instead, it provides that information along with the job status notification when the job status changes to ‘COMPLETE`. The only way to capture that notification is through an [Amazon Eventbridge](aws.amazon.com/eventbridge/) rule that forwards the a MediaWatch job status change on `COMPLETE` to another service, such as [CloudWatch Logs] (aws.amazon.com/cloudwatch/) log group

This adapter is written to get output information from a CloudWatch log group that has had MediaWatch complete events forwarded to it by an EventBridge group. The ‘setup!` method can be used to create these for you, at conventional names the adapter will be default use.

ActiveEncode::Base.engine_adapter.setup!

OR, there is experimental functionality to get what we can directly from the job without requiring a CloudWatch log – this is expected to be complete only for HLS output at present. It seems to work well for HLS output. To opt-in, and not require CloudWatch logs:

ActiveEncode::Base.engine_adapter.direct_output_lookup = true

## Example

ActiveEncode::Base.engine_adapter = :media_convert
ActiveEncode::Base.engine_adapter.role = 'arn:aws:iam::123456789012:role/service-role/MediaConvert_Default_Role'
ActiveEncode::Base.engine_adapter.output_bucket = 'output-bucket'

ActiveEncode::Base.engine_adapter.setup!

encode = ActiveEncode::Base.create(
  "file://path/to/file.mp4",
  {
    masterfile_bucket: "name-of-my-masterfile_bucket"
    output_prefix: "path/to/output/base_name_of_outputs",
    use_original_url: true,
    outputs: [
      { preset: "my-hls-preset-high", modifier: "_high" },
      { preset: "my-hls-preset-medium", modifier: "_medium" },
      { preset: "my-hls-preset-low", modifier: "_low" },
    ]
  }
)

## More info

A more detailed guide is available in the repo at [guides/media_convert_adapter.md](../../../guides/media_convert_adapter.md)

Defined Under Namespace

Classes: ResultsNotAvailable

Constant Summary collapse

JOB_STATES = 
{
  "SUBMITTED" => :running, "PROGRESSING" => :running, "CANCELED" => :cancelled,
  "ERROR" => :failed, "COMPLETE" => :completed
}.freeze
OUTPUT_GROUP_TEMPLATES = 
{
  hls: { min_segment_length: 0, segment_control: "SEGMENTED_FILES", segment_length: 10 },
  dash_iso: { fragment_length: 2, segment_control: "SEGMENTED_FILES", segment_length: 30 },
  file: {},
  ms_smooth: { fragment_length: 2 },
  cmaf: { fragment_length: 2, segment_control: "SEGMENTED_FILES", segment_length: 10 }
}.freeze
SETUP_LOG_GROUP_RETENTION_DAYS = 
3

Instance Attribute Summary collapse

Instance Method Summary collapse

Instance Attribute Details

#direct_output_lookup ⇒ Object

Returns the value of attribute direct_output_lookup.



110
111
112
 
# File 'lib/active_encode/engine_adapters/media_convert_adapter.rb', line 110

def direct_output_lookup
  @direct_output_lookup
end

#direct_output_lookup if true, do NOT get output information from cloudwatch, ⇒ Object

instead retrieve and construct it only from job itself. Currently working only for HLS output. default false.



110
 
# File 'lib/active_encode/engine_adapters/media_convert_adapter.rb', line 110

attr_accessor :role, :output_bucket, :direct_output_lookup

#log_group ⇒ Object 



252
253
254
 
# File 'lib/active_encode/engine_adapters/media_convert_adapter.rb', line 252

def log_group
  @log_group ||= "/aws/events/active-encode/mediaconvert/#{queue}"
end

#log_group log_group_name that is being used to capture output=(value) ⇒ Object (writeonly) 



114
 
# File 'lib/active_encode/engine_adapters/media_convert_adapter.rb', line 114

attr_writer :log_group, :queue

#output_bucket ⇒ Object

Returns the value of attribute output_bucket.



110
111
112
 
# File 'lib/active_encode/engine_adapters/media_convert_adapter.rb', line 110

def output_bucket
  @output_bucket
end

#output_bucket simple bucket name to write output to(simplebucketnametowriteoutputto) ⇒ Object 



110
 
# File 'lib/active_encode/engine_adapters/media_convert_adapter.rb', line 110

attr_accessor :role, :output_bucket, :direct_output_lookup

#queue ⇒ Object 



256
257
258
 
# File 'lib/active_encode/engine_adapters/media_convert_adapter.rb', line 256

def queue
  @queue ||= "Default"
end

#queue name of MediaConvert queue to use.=(value) ⇒ Object (writeonly) 



114
 
# File 'lib/active_encode/engine_adapters/media_convert_adapter.rb', line 114

attr_writer :log_group, :queue

#role ⇒ Object

Returns the value of attribute role.



110
111
112
 
# File 'lib/active_encode/engine_adapters/media_convert_adapter.rb', line 110

def role
  @role
end

#role simple name of AWS role to pass to MediaConvert, eg `my-role-name`(simplenameofAWSroletopasstoMediaConvert, eg`my-role-name`) ⇒ Object 



110
 
# File 'lib/active_encode/engine_adapters/media_convert_adapter.rb', line 110

attr_accessor :role, :output_bucket, :direct_output_lookup

Instance Method Details

#cancel(id) ⇒ Object 



247
248
249
250
 
# File 'lib/active_encode/engine_adapters/media_convert_adapter.rb', line 247

def cancel(id)
  mediaconvert.cancel_job(id: id)
  find(id)
end

#create(input_url, options = {}) ⇒ Object

Required options:

  • ‘output_prefix`: The S3 key prefix to use as the base for all outputs. Will be

    combined with configured `output_bucket` to be passed to MediaConvert
`destination`. Alternately see `destination` arg; one or the other
is required.

  • ‘destination`: The full s3:// URL to be passed to MediaConvert `destination` as output

    location an filename base.  `output_bucket` config is ignored if you
pass `destination`. Alternately see `output_prefix` arg; one or the
other is required.

  • ‘outputs`: An array of `modifier` options defining how to transcode and

    name the outputs. The "modifier" option will be passed as `name_modifier`
to AWS, to be added as a suffix on to `output_prefix` to create the
filenames for each output.

Optional options:

  • ‘masterfile_bucket`: All input will first be copied to this bucket, before being passed

    to MediaConvert. You can skip this copy by passing `use_original_url`
option, and an S3-based input. `masterfile_bucket` **is** required
unless use_original_url is true and an S3 input source.

  • ‘use_original_url`: If `true`, any S3 URL passed in as input will be passed directly to

    MediaConvert as the file input instead of copying the source to
the `masterfile_bucket`.

  • ‘media_type`: `audio` or `video`. Default `video`. Triggers use of a correspoinding

    template for arguments sent to AWS create_job API.

  • ‘output_type`: One of: `hls`, `dash_iso`, `file`, `ms_smooth`, `cmaf`. Default `hls`.

    Triggers use of a corresponding template for arguments sent to AWS
create_job API.

  • ‘output_group_destination_settings`: A hash of additional `destination_settings` to be

    sent to MediaConvert with the output_group.  Can include `s3_settings` key
with  `access_control` and `encryption` settings. See examples at:
https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/MediaConvert/Client.html#create_job-instance_method

Example: 

output_prefix: "path/to/output/files",
outputs: [
    {preset: "System-Avc_16x9_1080p_29_97fps_8500kbps", modifier: "-1080",
    "System-Avc_16x9_720p_29_97fps_5000kbps", modifier: "-720",
    "System-Avc_16x9_540p_29_97fps_3500kbps", modifier: "-540"
  ]
}

}



220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
 
# File 'lib/active_encode/engine_adapters/media_convert_adapter.rb', line 220

def create(input_url, options = {})
  input_url = s3_uri(input_url, options)

  input = options[:media_type] == :audio ? make_audio_input(input_url) : make_video_input(input_url)

  create_job_params = {
    queue: queue,
    role: role,
    settings: {
      inputs: [input],
      output_groups: make_output_groups(options)
    }
  }

  response = mediaconvert.create_job(create_job_params)
  job = response.job
  build_encode(job)
end

#find(id, _opts = {}) ⇒ Object 



239
240
241
242
243
244
245
 
# File 'lib/active_encode/engine_adapters/media_convert_adapter.rb', line 239

def find(id, _opts = {})
  response = mediaconvert.get_job(id: id)
  job = response.job
  build_encode(job)
rescue Aws::MediaConvert::Errors::NotFound
  raise ActiveEncode::NotFound, "Job #{id} not found"
end

#setup! ⇒ Object

Creates a [CloudWatch Logs] (aws.amazon.com/cloudwatch/) log group and an EventBridge rule to forward status change notifications to the log group, to catch result information from MediaConvert jobs.

Will use the configured ‘queue` and `log_group` values.

The active AWS user/role when calling the ‘#setup!` method will require permissions to create the necessary CloudWatch and EventBridge resources

This method chooses a conventional name for the EventBridge rule, if a rule by that name already exists, it will silently exit. So this method can be called in a boot process, to check if this infrastructure already exists, and create it only if it does not.



128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
 
# File 'lib/active_encode/engine_adapters/media_convert_adapter.rb', line 128

def setup!
  rule_name = "active-encode-mediaconvert-#{queue}"
  return true if event_rule_exists?(rule_name)

  queue_arn = mediaconvert.get_queue(name: queue).queue.arn

  event_pattern = {
    source: ["aws.mediaconvert"],
    "detail-type": ["MediaConvert Job State Change"],
    detail: {
      queue: [queue_arn],
      status: ["COMPLETE"]
    }
  }

  # AWS is inconsistent about whether a cloudwatch ARN has :* appended
  # to the end, and we need to make sure it doesn't in the rule target.
  log_group_arn = create_log_group(log_group).arn.chomp(":*")

  cloudwatch_events.put_rule(
    name: rule_name,
    event_pattern: event_pattern.to_json,
    state: "ENABLED",
    description: "Forward MediaConvert job state changes on COMPLETE from queue #{queue} to #{log_group}"
  )

  cloudwatch_events.put_targets(
    rule: rule_name,
    targets: [
      {
        id: "Id#{SecureRandom.uuid}",
        arn: log_group_arn
      }
    ]
  )
  true
end