puppet_metadata

License Test codecov Release RubyGem Version RubyGem Downloads Donated by Ewoud Kohl van Wijngaarden

The gem intends to provide an abstraction over Puppet's metadata.json file. Its API allow easy iteration over its illogical data structures.

New CLI interface in 6.0.0

Version 6.0.0 introduces a new CLI interface, in bin/puppet-metadata. It provides a new way of handling default CLI options, like the path to the metadata.json.

$ bundle exec bin/puppet-metadata --help
Usage: puppet-metadata [options] <action> [options]
        --filename METADATA          Metadata filename

ACTIONS
  os-versions         Manage operating system versions in metadata.json
  setfiles            Show the various setfiles supported by the metadata

See 'puppet-metadata ACTION --help' for more information on a specific action.

--filename is optional. If ommitted, a metadata.json in the current directory will be parsed.

Each action is implemented as a file in lib/puppet_metadata/command/*rb and automatically loaded via lib/puppet_metadata/command.rb.

Manage OS versions in metadata.json

The os-versions command provides a unified interface to view, add, and remove operating system versions in the metadata.json.

List supported OS versions

By default, os-versions shows which OS versions in your metadata.json are still supported and which are EOL:

$ bundle exec puppet-metadata os-versions
module-name supports these non-EOL operating system versions:
  AlmaLinux: 8, 9
  CentOS: 9
  Debian: 11, 12
  OracleLinux: 8, 9, 10
  RedHat: 8, 9
  Rocky: 8, 9
  Ubuntu: 22.04, 24.04

module-name supports these EOL operating system versions:
  Fedora: 40
  Ubuntu: 20.04

You can filter to a specific OS:

$ bundle exec puppet-metadata os-versions --os Ubuntu
module-name supports these non-EOL operating system versions:
  Ubuntu: 22.04, 24.04

module-name supports these EOL operating system versions:
  Ubuntu: 20.04

Add missing supported OS versions

Use --add-missing to automatically add all non-EOL OS versions to metadata.json:

$ bundle exec puppet-metadata os-versions --add-missing
Added support:
CentOS => 10
Debian => 13

These OSes are exceptions (to align with beaker-hostgenerator support):

  • For SLES, only major versions are added.
  • For Ubuntu, only LTS versions are added.

Remove EOL OS versions

Use --remove-eol to automatically remove all EOL OS versions from metadata.json:

$ bundle exec puppet-metadata os-versions --remove-eol
Removed EOL operating systems:
CentOS => 7, 8
Debian => 9, 10
Ubuntu => 20.04

You can preview changes without modifying metadata.json using --noop:

$ bundle exec puppet-metadata os-versions --add-missing --noop
[NOOP] Would add support:
CentOS => 10
Debian => 13

Generating Github Actions outputs

To get outputs usable in Github Actions, there is the metadata2gha command available. This generates based on metadata.json, such as Beaker setfiles, Puppet major versions and a Puppet unit test matrix.

$ metadata2gha
puppet_major_versions=[{"name":"Puppet 8","value":8,"collection":"puppet8"},{"name":"Puppet 7","value":7,"collection":"puppet7"}]
puppet_unit_test_matrix=[{"puppet":8,"ruby":"3.2"},{"puppet":7,"ruby":"2.7"}]
puppet_beaker_test_matrix=[{"name":"Puppet 8 - Debian 12","env":{"BEAKER_PUPPET_COLLECTION":"puppet8","BEAKER_SETFILE":"debian12-64{hostname=debian12-64-puppet8}"}},{"name":"Puppet 7 - Debian 12","env":{"BEAKER_PUPPET_COLLECTION":"puppet7","BEAKER_SETFILE":"debian12-64{hostname=debian12-64-puppet7}"}}]

Puppet major versions formatted for readability:

[
  {
    "name": "Puppet 8",
    "value": 8,
    "collection": "puppet8"
  },
  {
    "name": "Puppet 7",
    "value": 7,
    "collection": "puppet7"
  }
]

Puppet unit test matrix formatted for readability:

[
  {
    "puppet": 8,
    "ruby": "3.2"
  },
  {
    "puppet": 7,
    "ruby": "2.7"
  }
]

Beaker test matrix formatted for readability

[
  {
    "name": "Puppet 8 - Debian 12",
    "env": {
      "BEAKER_PUPPET_COLLECTION": "puppet8",
      "BEAKER_SETFILE": "debian12-64{hostname=debian12-64-puppet8}"
    }
  },
  {
    "name": "Puppet 7 - Debian 12",
    "env": {
      "BEAKER_PUPPET_COLLECTION": "puppet7",
      "BEAKER_SETFILE": "debian12-64{hostname=debian12-64-puppet7}"
    }
  }
]

If you need custom hostname or multiple hosts in your integration tests this could be achived by using the --beaker-hosts option

Option argument is 'HOSTNAME:ROLES;HOSTNAME:..;..' where

  • hosts are separated by ';'
  • host number and roles are separated by ':'
  • Roles follow beaker-hostgenerator syntax If you don't need any extra roles use '1;2;..'
$ metadata2gha --beaker-hosts 'foo:primary.ma;bar:secondary.a'

This results in the following JSON data

[
  {
    "name": "Puppet 7 - Debian 12",
    "env": {
      "BEAKER_PUPPET_COLLECTION": "puppet7",
      "BEAKER_SETFILE": "debian12-64primary.ma{hostname=foo-puppet7}-debian12-64secondary.a{hostname=bar-puppet7}"
    }
  }
]

If you need to Expand the matrix ie by product versions it could be achived by using the --beaker-facter option

Option argument is 'FACT:LABEL:VALUE,VALUE,..' where

  • Fact, label and values are separated by ':'
  • Values are separated by ','
$ metadata2gha --beaker-facter 'mongodb_repo_version:MongoDB:4.4,5.0,6.0,7.0'

This results in the following JSON data

[
  {
    "name": "Puppet 7 - Debian 12 - MongoDB 4.4",
    "env": {
        "BEAKER_PUPPET_COLLECTION": "puppet7",
        "BEAKER_SETFILE": "debian12-64{hostname=debian12-64-puppet7}",
        "BEAKER_FACTER_mongodb_repo_version": "4.4"
    }
  },
  {
    "name": "Puppet 7 - Debian 12 - MongoDB 5.0",
    "env": {
      "BEAKER_PUPPET_COLLECTION": "puppet7",
      "BEAKER_SETFILE": "debian12-64{hostname=debian12-64-puppet7}",
      "BEAKER_FACTER_mongodb_repo_version": "5.0"
    }
  },
  {
    "name": "Puppet 7 - Debian 12 - MongoDB 6.0",
    "env": {
      "BEAKER_PUPPET_COLLECTION": "puppet7",
      "BEAKER_SETFILE": "debian12-64{hostname=debian12-64-puppet7}",
      "BEAKER_FACTER_mongodb_repo_version": "6.0"
    }
  },
  {
    "name": "Puppet 7 - Debian 12 - MongoDB 7.0",
    "env": {
      "BEAKER_PUPPET_COLLECTION": "puppet7",
      "BEAKER_SETFILE": "debian12-64{hostname=debian12-64-puppet7}",
      "BEAKER_FACTER_mongodb_repo_version": "7.0"
    }
  }
]

Work with the API

The API can be initialised like this:

require 'puppet_metadata'
metadata = PuppetMetadata.read('/path/to/a/metadata.json')

The metadata object has several different methods that we can call

List all supported operating systems

[9] pry(main)> metadata.operatingsystems
=> {"Archlinux"=>nil, "Gentoo"=>nil, "Fedora"=>["32", "33"], "CentOS"=>["7", "8"], "RedHat"=>["7", "8"], "Ubuntu"=>["18.04"], "Debian"=>["9", "10"], "VirtuozzoLinux"=>["7"]}
[10] pry(main)>

List supported major puppet versions

[10] pry(main)> metadata.puppet_major_versions
=> [6, 7]
[11] pry(main)>

Check if an operating systems is supported

[6] pry(main)> metadata.os_release_supported?('Archlinux', nil)
=> true
[7] pry(main)>

Get all versions for an Operating System that are not EoL

[1] pry(main)> require 'puppet_metadata'
=> true
[2] pry(main)> PuppetMetadata::OperatingSystem.supported_releases('RedHat')
=> ["8", "9", "10"]
[3] pry(main)> PuppetMetadata::OperatingSystem.supported_releases('windows')
=> []
[4] pry(main)>

For Operating systems without any known releases, an empty array is returned.

Get all versions for an Operating System that are not EoL after a certain date

[1] pry(main)> require 'puppet_metadata'
=> true
[2] pry(main)> PuppetMetadata::OperatingSystem.supported_releases('CentOS', Date.parse('2025-04-15'))
=> ["9", "10"]
[3] pry(main)>

CentOS 8 and older aren't listed. 8 is EoL since 2024-05-31.

Updating OS EOL dates

The EOL dates for operating systems are stored in data/eol_dates.json and are automatically updated weekly via GitHub Actions using data from endoflife.date.

  • For Amazon Linux, this is security support, not standard support.
  • For Debian, this is extended life cycle, not standard support.
  • For CentOS, this is security support, not active support.
  • For OracleLinux, this is basic support, not extended support.
  • For RedHat, this is maintenance support, not extended life cycle.
  • For Rocky, this is security support, not active support.
  • For SLES, this is general support, not long term service pack support.

To manually update the EOL dates:

./bin/update_eol_dates

Adding new operating systems

To add a new operating system to the EOL tracking:

  1. Add an entry to the OS_MAPPING hash in bin/update_eol_dates
  2. Map it to the corresponding endoflife.date product identifier
  3. Run ./bin/update_eol_dates to fetch the data
  4. If the OS requires special handling (like Amazon Linux which uses multiple API endpoints), add a custom handler function

List supported setfiles

When running beaker on the CLI, you can specify a specific setfile. puppet_metadata provides bin/setfiles to list all setfiles:

$ bundle exec setfiles
Skipping EOL operating system Debian 10
Skipping EOL operating system Ubuntu 18.04
BEAKER_SETFILE="centos9-64{hostname=centos9-64-puppet8.example.com}"
BEAKER_SETFILE="centos9-64{hostname=centos9-64-puppet7.example.com}"
BEAKER_SETFILE="debian11-64{hostname=debian11-64-puppet8.example.com}"
BEAKER_SETFILE="debian11-64{hostname=debian11-64-puppet7.example.com}"
BEAKER_SETFILE="ubuntu2004-64{hostname=ubuntu2004-64-puppet8.example.com}"
BEAKER_SETFILE="ubuntu2004-64{hostname=ubuntu2004-64-puppet7.example.com}"

As an argument you can provide a path to a metadata.json. If none provided, it assumes that there's a metadata.json in the same directory where you run the command. To make copy and paste easier, each setfile string is prefixed so it can directly be used as environment variable.

Transfer Notice

This plugin was originally authored by Ewoud Kohl van Wijngaarden. The maintainer preferred that Vox Pupuli take ownership of the module for future improvement and maintenance. Existing pull requests and issues were transferred, please fork and continue to contribute here.

License

This gem is licensed under the Apache-2 license.

Release information

To make a new release, please do:

  • Update the version in the puppet_metadata.gemspec file
  • Install gems with bundle install --with release --path .vendor
  • generate the changelog with bundle exec rake changelog
  • Create a PR with it
  • After it got merged, push a tag. A github workflow will do the actual release