Class: GitHubChangelogGenerator::Fetcher

Inherits:
Object
  • Object
show all
Defined in:
lib/github_changelog_generator/fetcher.rb

Overview

A Fetcher responsible for all requests to GitHub and all basic manipulation with related data (such as filtering, validating, e.t.c)

Example: fetcher = GitHubChangelogGenerator::Fetcher.new options

Constant Summary collapse

PER_PAGE_NUMBER = 
30
CHANGELOG_GITHUB_TOKEN = 
"CHANGELOG_GITHUB_TOKEN"
GH_RATE_LIMIT_EXCEEDED_MSG = 
"Warning: Can't finish operation: GitHub API rate limit exceeded, change log may be " \
"missing some issues. You can limit the number of issues fetched using the `--max-issues NUM` argument."
NO_TOKEN_PROVIDED = 
"Warning: No token provided (-t option) and variable $CHANGELOG_GITHUB_TOKEN was not found. " \
"This script can make only 50 requests to GitHub API per hour without token!"

Instance Method Summary collapse

Constructor Details

#initialize(options = {}) ⇒ Fetcher

Returns a new instance of Fetcher.



16
17
18
19
20
21
22
23
24
25
26
27
 
# File 'lib/github_changelog_generator/fetcher.rb', line 16

def initialize(options = {})
  @options = options || {}
  @user = @options[:user]
  @project = @options[:project]
  @github_token = fetch_github_token
  @github_options = { per_page: PER_PAGE_NUMBER }
  @github_options[:oauth_token] = @github_token unless @github_token.nil?
  @github_options[:endpoint] = @options[:github_endpoint] unless @options[:github_endpoint].nil?
  @github_options[:site] = @options[:github_endpoint] unless @options[:github_site].nil?

  @github = check_github_response { Github.new @github_options }
end

Instance Method Details

#check_github_response ⇒ Object

This is wrapper with rescue block

Returns:

  • (Object) —

    returns exactly the same, what you put in the block, but wrap it with begin-rescue block



51
52
53
54
55
56
57
58
59
60
61
62
 
# File 'lib/github_changelog_generator/fetcher.rb', line 51

def check_github_response
  begin
    value = yield
  rescue Github::Error::Unauthorized => e
    Helper.log.error e.body.red
    abort "Error: wrong GitHub token"
  rescue Github::Error::Forbidden => e
    Helper.log.warn e.body.red
    Helper.log.warn GH_RATE_LIMIT_EXCEEDED_MSG.yellow
  end
  value
end

#fetch_closed_issues_and_pr ⇒ Tuple

This method fetch all closed issues and separate them to pull requests and pure issues (pull request is kind of issue in term of GitHub)

Returns:

  • (Tuple) —

    with (issues, pull-requests)



90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
 
# File 'lib/github_changelog_generator/fetcher.rb', line 90

def fetch_closed_issues_and_pr
  print "Fetching closed issues...\r" if @options[:verbose]
  issues = []

  begin
    response = @github.issues.list user: @options[:user],
                                   repo: @options[:project],
                                   state: "closed",
                                   filter: "all",
                                   labels: nil
    page_i = 0
    count_pages = response.count_pages
    response.each_page do |page|
      page_i += PER_PAGE_NUMBER
      print_in_same_line("Fetching issues... #{page_i}/#{count_pages * PER_PAGE_NUMBER}")
      issues.concat(page)
      break if @options[:max_issues] && issues.length >= @options[:max_issues]
    end
    print_empty_line
    Helper.log.info "Received issues: #{issues.count}"

  rescue
    Helper.log.warn GH_RATE_LIMIT_EXCEEDED_MSG.yellow
  end

  # separate arrays of issues and pull requests:
  issues.partition do |x|
    x[:pull_request].nil?
  end
end

#fetch_closed_pull_requests ⇒ Array

Fetch all pull requests. We need them to detect :merged_at parameter

Returns:

  • (Array) —

    all pull requests



123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
 
# File 'lib/github_changelog_generator/fetcher.rb', line 123

def fetch_closed_pull_requests
  pull_requests = []
  begin
    response = if @options[:release_branch].nil?
                 @github.pull_requests.list @options[:user],
                                            @options[:project],
                                            state: "closed"
               else
                 @github.pull_requests.list @options[:user],
                                            @options[:project],
                                            state: "closed",
                                            base: @options[:release_branch]
               end
    page_i = 0
    count_pages = response.count_pages
    response.each_page do |page|
      page_i += PER_PAGE_NUMBER
      log_string = "Fetching merged dates... #{page_i}/#{count_pages * PER_PAGE_NUMBER}"
      print_in_same_line(log_string)
      pull_requests.concat(page)
    end
    print_empty_line
  rescue
    Helper.log.warn GH_RATE_LIMIT_EXCEEDED_MSG.yellow
  end

  Helper.log.info "Fetching merged dates: #{pull_requests.count}"
  pull_requests
end

#fetch_commit(event) ⇒ Hash

Fetch commit for specified event

Returns:

  • (Hash) 


217
218
219
 
# File 'lib/github_changelog_generator/fetcher.rb', line 217

def fetch_commit(event)
  @github.git_data.commits.get @options[:user], @options[:project], event[:commit_id]
end

#fetch_date_of_tag(tag) ⇒ Time

Fetch tag time from repo

Parameters:

  • tag (Hash)

Returns:

  • (Time) —

    time of specified tag



203
204
205
206
207
208
209
210
211
212
213
 
# File 'lib/github_changelog_generator/fetcher.rb', line 203

def fetch_date_of_tag(tag)
  begin
    commit_data = @github.git_data.commits.get @options[:user],
                                               @options[:project],
                                               tag["commit"]["sha"]
  rescue
    Helper.log.warn GH_RATE_LIMIT_EXCEEDED_MSG.yellow
  end
  time_string = commit_data["committer"]["date"]
  Time.parse(time_string)
end

#fetch_events_async(issues) ⇒ Void

Fetch event for all issues and add them to :events

Parameters:

  • issues (Array)

Returns:

  • (Void) 


167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
 
# File 'lib/github_changelog_generator/fetcher.rb', line 167

def fetch_events_async(issues)
  i = 0
  max_thread_number = 50
  threads = []
  issues.each_slice(max_thread_number) do |issues_slice|
    issues_slice.each do |issue|
      threads << Thread.new do
        begin
          response = @github.issues.events.list user: @options[:user],
                                                repo: @options[:project],
                                                issue_number: issue["number"]
          issue[:events] = []
          response.each_page do |page|
            issue[:events].concat(page)
          end
        rescue
          Helper.log.warn GH_RATE_LIMIT_EXCEEDED_MSG.yellow
        end
        print_in_same_line("Fetching events for issues and PR: #{i + 1}/#{issues.count}")
        i += 1
      end
    end
    threads.each(&:join)
    threads = []
  end

  # to clear line from prev print
  print_empty_line

  Helper.log.info "Fetching events for issues and PR: #{i}"
end

#fetch_github_token ⇒ String

Returns GitHub token. First try to use variable, provided by –token option, otherwise try to fetch it from CHANGELOG_GITHUB_TOKEN env variable.

Returns:

  • (String) 


33
34
35
36
37
38
39
 
# File 'lib/github_changelog_generator/fetcher.rb', line 33

def fetch_github_token
  env_var = @options[:token] ? @options[:token] : (ENV.fetch CHANGELOG_GITHUB_TOKEN, nil)

  Helper.log.warn NO_TOKEN_PROVIDED.yellow unless env_var

  env_var
end

#get_all_tags ⇒ Array

Fetch all tags from repo

Returns:

  • (Array) —

    array of tags



43
44
45
46
47
 
# File 'lib/github_changelog_generator/fetcher.rb', line 43

def get_all_tags
  print "Fetching tags...\r" if @options[:verbose]

  check_github_response { github_fetch_tags }
end

#github_fetch_tags ⇒ Array

Fill input array with tags

Returns:

  • (Array) —

    array of tags in repo



66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
 
# File 'lib/github_changelog_generator/fetcher.rb', line 66

def github_fetch_tags
  tags = []
  response = @github.repos.tags @options[:user], @options[:project]
  page_i = 0
  count_pages = response.count_pages
  response.each_page do |page|
    page_i += PER_PAGE_NUMBER
    print_in_same_line("Fetching tags... #{page_i}/#{count_pages * PER_PAGE_NUMBER}")
    tags.concat(page)
  end
  print_empty_line

  if tags.count == 0
    Helper.log.warn "Warning: Can't find any tags in repo.\
Make sure, that you push tags to remote repo via 'git push --tags'".yellow
  else
    Helper.log.info "Found #{tags.count} tags"
  end
  tags
end

#print_empty_line ⇒ Object

Print long line with spaces on same line to clear prev message



160
161
162
 
# File 'lib/github_changelog_generator/fetcher.rb', line 160

def print_empty_line
  print_in_same_line("                                                                       ")
end

#print_in_same_line(log_string) ⇒ Object

Print specified line on the same string

Parameters:

  • log_string (String) 


155
156
157
 
# File 'lib/github_changelog_generator/fetcher.rb', line 155

def print_in_same_line(log_string)
  print log_string + "\r"
end