Module: LlmsTxt

Defined in:
lib/llms_txt.rb,
lib/llms_txt/cli.rb,
lib/llms_txt/config.rb,
lib/llms_txt/errors.rb,
lib/llms_txt/parser.rb,
lib/llms_txt/version.rb,
lib/llms_txt/generator.rb,
lib/llms_txt/validator.rb,
lib/llms_txt/bulk_transformer.rb,
lib/llms_txt/markdown_transformer.rb

Defined Under Namespace

Modules: Errors Classes: BulkTransformer, CLI, Config, Generator, MarkdownTransformer, ParsedContent, Parser, Validator

Constant Summary collapse

VERSION =

Current version of the LlmsTxt gem

'0.2.0'

Class Method Summary collapse

Class Method Details

.bulk_transform(docs_path, options = {}) ⇒ Array<String>

Bulk transforms multiple markdown files to be AI-friendly

Examples:

Bulk transform with direct options

LlmsTxt.bulk_transform('./docs',
  base_url: 'https://myproject.io',
  suffix: '.ai',
  excludes: ['**/private/**', 'draft-*.md']
)

Bulk transform using config file

LlmsTxt.bulk_transform('./docs', config_file: 'llms-txt.yml')

Parameters:

  • docs_path (String) —

    path to documentation directory

  • options (Hash) (defaults to: {}) —

    transformation options

Options Hash (options):

  • :config_file (String) —

    path to YAML config file (auto-finds llms-txt.yml)

  • :base_url (String) —

    base URL for expanding relative links (overrides config)

  • :convert_urls (Boolean) —

    convert HTML URLs to markdown format (overrides config)

  • :suffix (String) —

    suffix for transformed files (default: ‘.llm’, overrides config)

  • :excludes (Array<String>) —

    glob patterns for files to exclude (overrides config)

  • :verbose (Boolean) —

    enable verbose output (overrides config)

Returns:

  • (Array<String>) —

    paths of transformed files



107
108
109
110
111
112
 
# File 'lib/llms_txt.rb', line 107

def bulk_transform(docs_path, options = {})
  config = Config.new(options[:config_file])
  merged_options = config.merge_with_options(options)

  BulkTransformer.new(docs_path, merged_options).transform_all
end

.generate_from_docs(docs_path = nil, options = {}) ⇒ String

Generates llms.txt from existing markdown documentation

Examples:

Generate from docs directory

LlmsTxt.generate_from_docs('./docs')

Generate using config file

LlmsTxt.generate_from_docs(config_file: 'llms-txt.yml')

Generate with config file and overrides

LlmsTxt.generate_from_docs('./docs',
  config_file: 'my-config.yml',
  title: 'Override Title'
)

Parameters:

  • docs_path (String, nil) (defaults to: nil) —

    path to documentation directory or file (optional if config_file provided)

  • options (Hash) (defaults to: {}) —

    generation options

Options Hash (options):

  • :config_file (String) —

    path to YAML config file (auto-finds llms-txt.yml)

  • :base_url (String) —

    base URL for converting relative links (overrides config)

  • :title (String) —

    project title (auto-detected if not provided, overrides config)

  • :description (String) —

    project description (auto-detected if not provided, overrides config)

  • :output (String) —

    output file path (default: ‘llms.txt’, overrides config)

  • :convert_urls (Boolean) —

    convert HTML URLs to markdown format (overrides config)

  • :verbose (Boolean) —

    enable verbose output (overrides config)

Returns:

  • (String) —

    generated llms.txt content



41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
 
# File 'lib/llms_txt.rb', line 41

def generate_from_docs(docs_path = nil, options = {})
  # Support config-first usage: generate_from_docs(config_file: 'path.yml')
  if docs_path.is_a?(Hash) && docs_path.key?(:config_file)
    options = docs_path
    docs_path = nil
  end

  config = Config.new(options[:config_file])
  merged_options = config.merge_with_options(options)

  # Use docs_path param or config file docs setting
  final_docs_path = docs_path || merged_options[:docs]

  Generator.new(final_docs_path, merged_options).generate
end

.parse(file_path) ⇒ Parser

Parses an existing llms.txt file

Parameters:

  • file_path (String) —

    path to the llms.txt file to parse

Returns:

  • (Parser) —

    parsed llms.txt object



118
119
120
 
# File 'lib/llms_txt.rb', line 118

def parse(file_path)
  Parser.new(file_path).parse
end

.transform_markdown(file_path, options = {}) ⇒ String

Transforms a markdown file to be AI-friendly

Examples:

Transform with direct options

LlmsTxt.transform_markdown('README.md',
  base_url: 'https://myproject.io',
  convert_urls: true
)

Transform using config file

LlmsTxt.transform_markdown('README.md', config_file: 'llms-txt.yml')

Parameters:

  • file_path (String) —

    path to markdown file

  • options (Hash) (defaults to: {}) —

    transformation options

Options Hash (options):

  • :config_file (String) —

    path to YAML config file (auto-finds llms-txt.yml)

  • :base_url (String) —

    base URL for expanding relative links (overrides config)

  • :convert_urls (Boolean) —

    convert HTML URLs to markdown format (overrides config)

  • :verbose (Boolean) —

    enable verbose output (overrides config)

Returns:

  • (String) —

    transformed markdown content



76
77
78
79
80
81
 
# File 'lib/llms_txt.rb', line 76

def transform_markdown(file_path, options = {})
  config = Config.new(options[:config_file])
  merged_options = config.merge_with_options(options)

  MarkdownTransformer.new(file_path, merged_options).transform
end

.validate(content) ⇒ Boolean

Validates llms.txt content

Parameters:

  • content (String) —

    the llms.txt content to validate

Returns:

  • (Boolean) —

    true if content is valid, false otherwise



126
127
128
 
# File 'lib/llms_txt.rb', line 126

def validate(content)
  Validator.new(content).valid?
end