Class: Aws::Firehose::Types::ParquetSerDe

Inherits:
Struct
  • Object
show all
Includes:
Structure
Defined in:
lib/aws-sdk-firehose/types.rb

Overview

Note:

When making an API call, you may pass ParquetSerDe data as a hash:

{
  block_size_bytes: 1,
  page_size_bytes: 1,
  compression: "UNCOMPRESSED", # accepts UNCOMPRESSED, GZIP, SNAPPY
  enable_dictionary_compression: false,
  max_padding_bytes: 1,
  writer_version: "V1", # accepts V1, V2
}

A serializer to use for converting data to the Parquet format before storing it in Amazon S3. For more information, see [Apache Parquet].

[1]: parquet.apache.org/documentation/latest/

Constant Summary collapse

SENSITIVE =
[]

Instance Attribute Summary collapse

Instance Attribute Details

#block_size_bytesInteger

The Hadoop Distributed File System (HDFS) block size. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is 256 MiB and the minimum is 64 MiB. Kinesis Data Firehose uses this value for padding calculations.

Returns:

  • (Integer)

3910
3911
3912
3913
3914
3915
3916
3917
3918
3919
# File 'lib/aws-sdk-firehose/types.rb', line 3910

class ParquetSerDe < Struct.new(
  :block_size_bytes,
  :page_size_bytes,
  :compression,
  :enable_dictionary_compression,
  :max_padding_bytes,
  :writer_version)
  SENSITIVE = []
  include Aws::Structure
end

#compressionString

The compression code to use over data blocks. The possible values are `UNCOMPRESSED`, `SNAPPY`, and `GZIP`, with the default being `SNAPPY`. Use `SNAPPY` for higher decompression speed. Use `GZIP` if the compression ratio is more important than speed.

Returns:

  • (String)

3910
3911
3912
3913
3914
3915
3916
3917
3918
3919
# File 'lib/aws-sdk-firehose/types.rb', line 3910

class ParquetSerDe < Struct.new(
  :block_size_bytes,
  :page_size_bytes,
  :compression,
  :enable_dictionary_compression,
  :max_padding_bytes,
  :writer_version)
  SENSITIVE = []
  include Aws::Structure
end

#enable_dictionary_compressionBoolean

Indicates whether to enable dictionary compression.

Returns:

  • (Boolean)

3910
3911
3912
3913
3914
3915
3916
3917
3918
3919
# File 'lib/aws-sdk-firehose/types.rb', line 3910

class ParquetSerDe < Struct.new(
  :block_size_bytes,
  :page_size_bytes,
  :compression,
  :enable_dictionary_compression,
  :max_padding_bytes,
  :writer_version)
  SENSITIVE = []
  include Aws::Structure
end

#max_padding_bytesInteger

The maximum amount of padding to apply. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is 0.

Returns:

  • (Integer)

3910
3911
3912
3913
3914
3915
3916
3917
3918
3919
# File 'lib/aws-sdk-firehose/types.rb', line 3910

class ParquetSerDe < Struct.new(
  :block_size_bytes,
  :page_size_bytes,
  :compression,
  :enable_dictionary_compression,
  :max_padding_bytes,
  :writer_version)
  SENSITIVE = []
  include Aws::Structure
end

#page_size_bytesInteger

The Parquet page size. Column chunks are divided into pages. A page is conceptually an indivisible unit (in terms of compression and encoding). The minimum value is 64 KiB and the default is 1 MiB.

Returns:

  • (Integer)

3910
3911
3912
3913
3914
3915
3916
3917
3918
3919
# File 'lib/aws-sdk-firehose/types.rb', line 3910

class ParquetSerDe < Struct.new(
  :block_size_bytes,
  :page_size_bytes,
  :compression,
  :enable_dictionary_compression,
  :max_padding_bytes,
  :writer_version)
  SENSITIVE = []
  include Aws::Structure
end

#writer_versionString

Indicates the version of row format to output. The possible values are `V1` and `V2`. The default is `V1`.

Returns:

  • (String)

3910
3911
3912
3913
3914
3915
3916
3917
3918
3919
# File 'lib/aws-sdk-firehose/types.rb', line 3910

class ParquetSerDe < Struct.new(
  :block_size_bytes,
  :page_size_bytes,
  :compression,
  :enable_dictionary_compression,
  :max_padding_bytes,
  :writer_version)
  SENSITIVE = []
  include Aws::Structure
end