Method: Aws::Kinesis::Client#put_records

Defined in:

lib/aws-sdk-kinesis/client.rb

#put_records(params = {}) ⇒ Types::PutRecordsOutput

Writes multiple data records into a Kinesis data stream in a single call (also referred to as a ‘PutRecords` request). Use this operation to send data into the stream for data ingestion and processing.

<note markdown=“1”> When invoking this API, you must use either the ‘StreamARN` or the `StreamName` parameter, or both. It is recommended that you use the `StreamARN` input parameter when you invoke this API.

</note>

Each ‘PutRecords` request can support up to 500 records. Each record in the request can be as large as 1 MiB, up to a limit of 5 MiB for the entire request, including partition keys. Each shard can support writes up to 1,000 records per second, up to a maximum data write total of 1 MiB per second.

You must specify the name of the stream that captures, stores, and transports the data; and an array of request ‘Records`, with each record in the array requiring a partition key and data blob. The record size limit applies to the total size of the partition key and data blob.

The data blob can be any type of data; for example, a segment from a log file, geographic/location data, website clickstream data, and so on.

The partition key is used by Kinesis Data Streams as input to a hash function that maps the partition key and associated data to a specific shard. An MD5 hash function is used to map partition keys to 128-bit integer values and to map associated data records to shards. As a result of this hashing mechanism, all data records with the same partition key map to the same shard within the stream. For more information, see [Adding Data to a Stream] in the *Amazon Kinesis Data Streams Developer Guide*.

Each record in the ‘Records` array may include an optional parameter, `ExplicitHashKey`, which overrides the partition key to shard mapping. This parameter allows a data producer to determine explicitly the shard where the record is stored. For more information, see [Adding Multiple Records with PutRecords] in the *Amazon Kinesis Data Streams Developer Guide*.

The ‘PutRecords` response includes an array of response `Records`. Each record in the response array directly correlates with a record in the request array using natural ordering, from the top to the bottom of the request and response. The response `Records` array always includes the same number of records as the request array.

The response ‘Records` array includes both successfully and unsuccessfully processed records. Kinesis Data Streams attempts to process all records in each `PutRecords` request. A single record failure does not stop the processing of subsequent records. As a result, PutRecords doesn’t guarantee the ordering of records. If you need to read records in the same order they are written to the stream, use PutRecord instead of ‘PutRecords`, and write to the same shard.

A successfully processed record includes ‘ShardId` and `SequenceNumber` values. The `ShardId` parameter identifies the shard in the stream where the record is stored. The `SequenceNumber` parameter is an identifier assigned to the put record, unique to all records in the stream.

An unsuccessfully processed record includes ‘ErrorCode` and `ErrorMessage` values. `ErrorCode` reflects the type of error and can be one of the following values: `ProvisionedThroughputExceededException` or `InternalFailure`. `ErrorMessage` provides more detailed information about the `ProvisionedThroughputExceededException` exception including the account ID, stream name, and shard ID of the record that was throttled. For more information about partially successful responses, see [Adding Multiple Records with PutRecords] in the *Amazon Kinesis Data Streams Developer Guide*.

After you write a record to a stream, you cannot modify that record or its order within the stream.

By default, data records are accessible for 24 hours from the time that they are added to a stream. You can use IncreaseStreamRetentionPeriod or DecreaseStreamRetentionPeriod to modify this retention period.

[1]: docs.aws.amazon.com/kinesis/latest/dev/developing-producers-with-sdk.html#kinesis-using-sdk-java-add-data-to-stream [2]: docs.aws.amazon.com/kinesis/latest/dev/developing-producers-with-sdk.html#kinesis-using-sdk-java-putrecords [3]: docs.aws.amazon.com/kinesis/latest/dev/kinesis-using-sdk-java-add-data-to-stream.html#kinesis-using-sdk-java-putrecords

Examples:

Request syntax with placeholder values

resp = client.put_records({
  records: [ # required
    {
      data: "data", # required
      explicit_hash_key: "HashKey",
      partition_key: "PartitionKey", # required
    },
  ],
  stream_name: "StreamName",
  stream_arn: "StreamARN",
})

Response structure

resp.failed_record_count #=> Integer
resp.records #=> Array
resp.records[0].sequence_number #=> String
resp.records[0].shard_id #=> String
resp.records[0].error_code #=> String
resp.records[0].error_message #=> String
resp.encryption_type #=> String, one of "NONE", "KMS"

Parameters:

• params (Hash) (defaults to: {}) —

  ({})

Options Hash (params):

• :records (required, Array<Types::PutRecordsRequestEntry>) —

  The records associated with the request.

• :stream_name (String) —

  The stream name associated with the request.

• :stream_arn (String) —

  The ARN of the stream.

Returns:

• (Types::PutRecordsOutput) —

  Returns a response object which responds to the following methods:

  • #failed_record_count => Integer

  • #records => Array&lt;Types::PutRecordsResultEntry&gt;

  • #encryption_type => String

See Also:

• AWS API Documentation

# File 'lib/aws-sdk-kinesis/client.rb', line 2344

def put_records(params = {}, options = {})
  req = build_request(:put_records, params)
  req.send_request(options)
end