Module: HasGeoLookup

Extended by:
ActiveSupport::Concern
Included in:
Geoname
Defined in:
lib/has_geo_lookup/concern.rb,
lib/has_geo_lookup.rb,
lib/has_geo_lookup/railtie.rb,
lib/has_geo_lookup/version.rb,
lib/has_geo_lookup/index_checker.rb,
lib/has_geo_lookup/boundary_importer.rb,
lib/generators/has_geo_lookup/install_generator.rb

Overview

app/models/concerns/has_geo_lookup.rb

Comprehensive geographic lookup functionality using GeoBoundaries.org and Geonames.org data

This concern provides both distance-based lookups (using Geonames.org) and precise geometric containment queries (using GeoBoundaries.org) for models with latitude/longitude coordinates. It includes data coverage utilities, municipal name cleaning, and comparison tools for geographic data quality analysis.

Examples:

Basic usage

class Listing < ApplicationRecord
  include HasGeoLookup
end

listing = Listing.first
listing.nearest_geonames(feature_class: "P", limit: 3)
listing.containing_boundaries
listing.compare_geo_sources

See Also:

Defined Under Namespace

Modules: DataCoverage, Generators Classes: BoundaryImporter, Error, GeoboundaryResult, IndexChecker, Railtie, Result

Constant Summary collapse

VERSION = 
"0.2.3"

Instance Method Summary collapse

Instance Method Details

#closest_county_or_parish(radius_km: 50, country_code: nil) ⇒ Object

Looks up the closest county or parish-level area using feature_code = ADM2.

Options:

:radius_km — limit search radius (default: 50)
:country_code — optionally restrict by country

Returns a Result struct with distance and matched geoname.



128
129
130
131
132
133
134
135
136
 
# File 'lib/has_geo_lookup/concern.rb', line 128

def closest_county_or_parish(radius_km: 50, country_code: nil)
  nearest_geonames(
    feature_class: "A",
    feature_code: "ADM2",
    radius_km: radius_km,
    # country_code: country_code,
    limit: 1
  ).first
end

#closest_metro ⇒ Object

Deprecated.

Use #within_metro instead

Legacy method for backward compatibility - will be removed after migration



342
343
344
 
# File 'lib/has_geo_lookup/concern.rb', line 342

def closest_metro
  within_metro
end

#closest_subdivision(radius_km: 1, country_code: nil) ⇒ Object

Looks up the closest subdivision (e.g., neighborhood or district) Defaults to feature_code = "PPLX" and 1 km radius

Options:

:radius_km — limit search radius (default: 1)
:country_code — optionally restrict by country

Returns a Result struct with distance and matched geoname.



146
147
148
149
150
151
152
153
154
 
# File 'lib/has_geo_lookup/concern.rb', line 146

def closest_subdivision(radius_km: 1, country_code: nil)
  nearest_geonames(
    feature_class: "P",
    feature_code: "PPLX",
    radius_km: radius_km,
    # country_code: country_code,
    limit: 1
  ).first
end

#closest_township(radius_km: 25, country_code: nil) ⇒ Object

Looks up the closest township-level area using feature_code = ADM3. Falls back to ADM4, then ADM5 if no ADM3 match is found.

Options:

:radius_km — limit search radius (default: 25)
:country_code — optionally restrict by country

Returns a Result struct with distance and matched geoname.



164
165
166
167
168
169
170
171
172
173
174
175
176
177
 
# File 'lib/has_geo_lookup/concern.rb', line 164

def closest_township(radius_km: 25, country_code: nil)
  %w[ADM3 ADM4 ADM5].each do |code|
    result = nearest_geonames(
      feature_class: "A",
      feature_code: code,
      radius_km: radius_km,
      # country_code: country_code,
      limit: 1
    ).first
    return result if result
  end

  nil
end

#compare_geo_sources ⇒ String

Compare geographic data from multiple sources for maintenance and debugging

This method displays a side-by-side comparison of geographic attribute values from different data sources: current stored values, GeoBoundaries.org data, and Geonames.org data. Apps can extend this by overriding additional_source_columns to add their own data sources (e.g., original API data).

Examples:

listing.compare_geo_sources
# => Displays formatted table comparing all geographic attributes across sources

Returns:

  • Formatted comparison table or error message if no coordinates



358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
 
# File 'lib/has_geo_lookup/concern.rb', line 358

def compare_geo_sources
  return "No coordinates available for comparison" unless latitude && longitude

  geo_attributes = %w[city county_or_parish state_or_province township subdivision_name country postal_code]
  
  # Collect all data first to avoid jumbled SQL output
  print "Collecting geographic data..."
  data_rows = []
  
  geo_attributes.each do |attr|
    print "."
    current_val = send(attr)
    boundary_val = get_boundary_value(attr)
    geoname_val = get_geoname_value(attr)
    
    # Get additional source columns from the implementing model
    additional_sources = respond_to?(:additional_source_columns, true) ? additional_source_columns(attr) : {}
    
    # Truncate long values for display
    current_display = truncate_value(current_val)
    boundary_display = truncate_value(boundary_val)
    geoname_display = truncate_value(geoname_val)
    
    # Check if current value differs from core sources and any additional sources
    all_source_vals = [boundary_val, geoname_val] + additional_sources.values
    marker = all_source_vals.none? { |val| current_val == val } ? " *" : ""
    
    row_data = {
      attr: attr.upcase,
      current: current_display || "(nil)",
      boundary: boundary_display || "(nil)",
      geoname: geoname_display || "(nil)",
      marker: marker
    }
    
    # Add additional source columns
    additional_sources.each do |column_name, value|
      row_data[column_name.to_sym] = truncate_value(value) || "(nil)"
    end
    
    data_rows << row_data
  end
  
  puts " done!\n"
  
  # Build header columns
  base_columns = %w[ATTRIBUTE CURRENT BOUNDARY GEONAMES]
  additional_columns = respond_to?(:additional_source_columns, true) ? 
    additional_source_columns(geo_attributes.first)&.keys || [] : []
  all_columns = base_columns + additional_columns.map(&:upcase)
  
  # Calculate total width
  total_width = all_columns.length * 20 + 4
  
  # Display results
  puts "\n" + "=" * total_width
  puts "GEO SOURCES COMPARISON"
  puts "Coordinates: #{latitude}, #{longitude}"
  puts "=" * total_width
  puts sprintf((["%-20s"] * all_columns.length).join(" "), *all_columns)
  puts "-" * (total_width + all_columns.length - 1)
  
  data_rows.each do |row|
    values = all_columns.map { |col| row[col.downcase.to_sym] }
    puts sprintf((["%-20s"] * values.length).join(" "), *values) + row[:marker]
  end
  
  puts "-" * (total_width + all_columns.length - 1)
  puts "* = Current value differs from all sources"
  puts "\nLegend:"
  puts "  CURRENT  - Value currently stored in database"
  puts "  BOUNDARY - Value from GeoBoundaries.org (precise polygon containment)"
  puts "  GEONAMES - Value from Geonames.org (nearest feature lookup)"
  
  # Add legend entries for additional sources
  if respond_to?(:additional_source_legend, true)
    additional_source_legend.each do |column, description|
      puts "  #{column.upcase.ljust(8)} - #{description}"
    end
  end
  
  puts "=" * total_width
  
  nil
end

#containing_boundaries(levels: nil) ⇒ Array<Geoboundary>

Find administrative boundaries that contain this point using precise geometric containment

This method uses PostGIS spatial operations to determine which GeoBoundaries.org administrative boundaries contain the current coordinate point. It performs automatic coordinate validation and swapping for common data issues.

Examples:

Find all containing boundaries

listing.containing_boundaries

Find only state and county level

listing.containing_boundaries(levels: ["ADM1", "ADM2"])

Parameters:

  • (defaults to: nil)

    Specific ADM levels to search (e.g., ["ADM1", "ADM2"]) If nil, searches all available levels

Returns:

  • Array of boundary records that contain this point, ordered by administrative level



198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
 
# File 'lib/has_geo_lookup/concern.rb', line 198

def containing_boundaries(levels: nil)
  return [] unless latitude && longitude

  # Validate coordinate ranges
  lat = latitude.to_f
  lng = longitude.to_f
  
  # Check if coordinates are swapped (common data issue)
  if !lat.between?(-90, 90)
    # Try swapping if latitude is invalid but longitude could be a valid latitude
    if lng.between?(-90, 90) && lat.between?(-180, 180)
      lat, lng = lng, lat
    else
      # Invalid coordinates that can't be fixed by swapping
      return []
    end
  elsif !lng.between?(-180, 180)
    # Longitude is invalid but latitude is valid - this is unusual, reject
    return []
  end

  # MySQL spatial functions expect POINT(latitude longitude) format (different from PostGIS)
  point_wkt = "POINT(#{lat} #{lng})"
  
  query = Geoboundary.where(
    "ST_Contains(boundary, ST_GeomFromText(?, 4326))",
    point_wkt
  )
  
  
  query = query.where(level: levels) if levels
  
  # Add limit to prevent memory issues on large datasets
  query.order(:level).limit(50)
rescue => e
  Rails.logger&.error "Spatial query failed for coordinates #{lat}, #{lng}: #{e.message}"
  []
end

#containing_boundary(level) ⇒ Object

Returns the boundary at a specific level that contains this point



238
239
240
 
# File 'lib/has_geo_lookup/concern.rb', line 238

def containing_boundary(level)
  containing_boundaries(levels: level).first
end

#county_or_parish_boundary ⇒ Object

GeoBoundary equivalent of closest_county_or_parish Returns the ADM2 boundary that contains this point, with fallback to closest geoname



244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
 
# File 'lib/has_geo_lookup/concern.rb', line 244

def county_or_parish_boundary
  # First try exact containment
  boundary = containing_boundary('ADM2')
  return GeoboundaryResult.new(boundary, 0.0, 'ADM2', boundary.name) if boundary
  
  # Fallback to closest geoname approach
  geoname_result = closest_county_or_parish
  if geoname_result && geoname_result.record
    # Try coordinate bridge: use geoname coordinates to find GeoBoundaries match
    # This improves language consistency (e.g., "Lisbon" → "Lisboa")
    geoname_lat = geoname_result.record.latitude
    geoname_lng = geoname_result.record.longitude
    
    if geoname_lat && geoname_lng
      # Create temporary checker at geoname coordinates
      temp_checker = Object.new.extend(HasGeoLookup)
      temp_checker.define_singleton_method(:latitude) { geoname_lat }
      temp_checker.define_singleton_method(:longitude) { geoname_lng }
      
      bridge_boundary = temp_checker.containing_boundary('ADM2')
      if bridge_boundary
        return GeoboundaryResult.new(bridge_boundary, geoname_result.distance_km, 'ADM2', bridge_boundary.name)
      end
    end
    
    # If coordinate bridge fails, use original geoname result
    return geoname_result
  end
  
  nil
end

#nearest_geonames(feature_class: nil, feature_code: nil, keyword: nil, radius_km: 100, limit: 5) ⇒ Array<Result>

Find nearby Geonames of a given type within a specified radius

This method performs distance-based lookup of geographic features from the Geonames.org dataset. It supports filtering by feature type and uses bounding box optimization for better performance on large datasets.

Examples:

Find closest populated places

listing.nearest_geonames(feature_class: "P", radius_km: 50)

Find administrative divisions by keyword

listing.nearest_geonames(keyword: "county", limit: 1)

Parameters:

  • (defaults to: nil)

    Geoname feature class (e.g., "P" for populated places)

  • (defaults to: nil)

    Geoname feature code (e.g., "PPL" for populated place)

  • (defaults to: nil)

    Search feature names/descriptions to auto-determine criteria

  • (defaults to: 100)

    Search radius in kilometers (default: 100)

  • (defaults to: 5)

    Maximum results to return (default: 5)

Returns:

  • Array of Result structs with :record, :distance_km, :feature_class, :feature_code



74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
 
# File 'lib/has_geo_lookup/concern.rb', line 74

def nearest_geonames(feature_class: nil, feature_code: nil, keyword: nil, radius_km: 100, limit: 5)
  return [] unless latitude && longitude

  if keyword && (feature_class.nil? || feature_code.nil?)
    fc = find_feature_class_and_code_by_keyword(keyword)
    feature_class ||= fc&.first
    feature_code  ||= fc&.last
  end

  return [] unless feature_code || feature_class

  # Calculate rough bounding box for fast filtering before expensive distance calculations
  # 1 degree ≈ 111km, so calculate degree offset for the radius
  lat_offset = radius_km / 111.0
  lng_offset = radius_km / (111.0 * Math.cos(Math::PI * latitude / 180.0))

  query = Geoname.where.not(latitude: nil, longitude: nil)

  query = query.where(feature_code: feature_code)         if feature_code
  query = query.where(feature_class: feature_class)       if feature_class

  # Add bounding box filter first (uses indexes, very fast)
  query = query.where(
    latitude: (latitude - lat_offset)..(latitude + lat_offset),
    longitude: (longitude - lng_offset)..(longitude + lng_offset)
  )

  query = query.select(<<~SQL.squish)
    geonames.*,
    (6371 * acos(
      cos(radians(#{latitude}))
      * cos(radians(latitude))
      * cos(radians(longitude) - radians(#{longitude}))
      + sin(radians(#{latitude}))
      * sin(radians(latitude))
    )) AS distance_km
  SQL

  query = query.having("distance_km <= ?", radius_km)
               .order("distance_km ASC")
               .limit(limit)

  query.map do |record|
    Result.new(record, record.try(:distance_km).to_f, feature_class, feature_code)
  end
end

#state_or_province_boundary ⇒ Object

GeoBoundary equivalent for state/province Returns the ADM1 boundary that contains this point



294
295
296
297
298
299
300
 
# File 'lib/has_geo_lookup/concern.rb', line 294

def state_or_province_boundary
  boundary = containing_boundary('ADM1')
  return GeoboundaryResult.new(boundary, 0.0, 'ADM1', boundary.name) if boundary
  
  # No geoname fallback for ADM1 since closest_ doesn't handle it
  nil
end

#subdivision_with_boundary_context ⇒ Object

For subdivision, we still use geonames since geoBoundaries doesn't have neighborhood-level data but we can add boundary context



304
305
306
307
308
309
310
311
312
313
 
# File 'lib/has_geo_lookup/concern.rb', line 304

def subdivision_with_boundary_context
  geoname_result = closest_subdivision
  return geoname_result unless geoname_result
  
  # Add boundary context to help validate the subdivision
  boundaries = containing_boundaries(levels: %w[ADM2 ADM3 ADM4 ADM5])
  geoname_result.record.define_singleton_method(:containing_boundaries) { boundaries }
  
  geoname_result
end

#township_boundary ⇒ Object

GeoBoundary equivalent of closest_township
Returns ADM3, ADM4 or ADM5 boundary that contains this point, with fallback



278
279
280
281
282
283
284
285
286
287
288
289
290
 
# File 'lib/has_geo_lookup/concern.rb', line 278

def township_boundary
  # Try ADM3 first, then ADM4, then ADM5
  %w[ADM5 ADM4 ADM3].each do |level|
    boundary = containing_boundary(level)
    return GeoboundaryResult.new(boundary, 0.0, level, boundary.name) if boundary
  end
  
  # Fallback to closest geoname approach
  geoname_result = closest_township
  return geoname_result if geoname_result
  
  nil
end

#validate_and_convert_coordinates(lat, lng, expected_country = nil) ⇒ Array<(Float, Float)>

Note:

This method requires PostGIS tables (geoboundaries, geonames) for country validation. In test environments or when PostGIS is unavailable, country validation is skipped.

Validate and convert coordinates from radians to degrees if needed

This method intelligently detects whether coordinates are provided in radians or degrees using a multi-step validation process:

  1. Range Check: If coordinates are outside degree ranges (|lat| > 90 or |lng| > 180), they are assumed to be radians and converted, unless they exceed reasonable bounds (> 1000)
  2. Country Validation: For ambiguous coordinates within degree ranges, attempts to validate against expected country boundaries using PostGIS spatial queries
  3. Fallback: If validation fails or PostGIS is unavailable, defaults to treating coordinates as degrees

This is particularly useful when dealing with data sources that may inconsistently provide coordinates in different units.

Examples:

Converting obvious radians to degrees

validate_and_convert_coordinates(0.7128, -1.2915, "US")
# => [40.8355, -74.0060] (converted from radians using country validation)

Preserving valid degrees

validate_and_convert_coordinates(40.7128, -74.0060, "US") 
# => [40.7128, -74.0060] (already in degrees, no conversion needed)

Handling coordinates outside degree ranges

validate_and_convert_coordinates(95.0, 1.5, "US")
# => [5441.5, 85.9] (lat > 90°, so both coordinates converted from radians)

Rejecting invalid coordinates

validate_and_convert_coordinates(2000.0, 3000.0, "US")
# => [nil, nil] (values too large to be reasonable coordinates)

Parameters:

  • Latitude coordinate in degrees or radians

  • Longitude coordinate in degrees or radians

  • (defaults to: nil)

    Optional 2-letter ISO country code (e.g., "US", "FR") used for boundary validation when coordinates are ambiguous

Returns:

  • Array containing [latitude, longitude] in degrees, or [nil, nil] if coordinates are invalid or outside reasonable bounds

See Also:

  • for details on boundary validation logic

Since:

  • 1.0.0



488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
 
# File 'lib/has_geo_lookup/concern.rb', line 488

def validate_and_convert_coordinates(lat, lng, expected_country = nil)
  return [nil, nil] unless lat && lng
  
  lat = lat.to_f
  lng = lng.to_f
  
  # First check: are coordinates obviously in radians? (outside degree ranges)
  if lat.abs > 90 || lng.abs > 180
    # Check if they could be reasonable radians (not absurdly large)  
    # Reasonable upper bound: 1000 (much larger than any reasonable coordinate)
    if lat.abs <= 1000 && lng.abs <= 1000
      # Assume they are radians and convert
      lat_deg = lat * 180.0 / Math::PI
      lng_deg = lng * 180.0 / Math::PI
      return [lat_deg, lng_deg]
    else
      # Values too large to be reasonable coordinates in any format
      return [nil, nil]
    end
  end
  
  # Coordinates are within degree ranges - but could still be radians
  # Use country validation to determine which is correct
  if expected_country.present?
    # Test as degrees first
    degrees_valid = coordinates_match_country?(lat, lng, expected_country)
    
    # If degrees don't match, try converting from radians
    unless degrees_valid
      # Check if coordinates could be radians (within radian range)
      if lat.abs <= Math::PI && lng.abs <= Math::PI
        lat_from_radians = lat * 180.0 / Math::PI
        lng_from_radians = lng * 180.0 / Math::PI
        
        # Test if radians-to-degrees conversion matches expected country
        if coordinates_match_country?(lat_from_radians, lng_from_radians, expected_country)
          return [lat_from_radians, lng_from_radians]
        end
      end
    end
  end
  
  # Default: assume coordinates are already in degrees
  [lat, lng]
end

#within_metro ⇒ Metro?

Find the metro area that contains this point using precise geometric containment

This method uses PostGIS spatial operations to determine which metropolitan area contains the current coordinate point. Metros are defined as collections of geoboundaries (administrative boundaries) that together form a cohesive region.

Examples:

listing.within_metro
# => #<Metro id: 1, name: "Bay Area", details: "San Francisco Bay Area">
listing.within_metro
# => nil (if coordinates are not within any defined metro area)

Returns:

  • Metro area that contains this point, or nil if not within any metro



330
331
332
333
334
335
336
337
338
 
# File 'lib/has_geo_lookup/concern.rb', line 330

def within_metro
  return nil unless latitude && longitude

  # Direct spatial containment query - check if this point is within any metro's boundaries
  # MySQL spatial functions expect POINT(latitude longitude) format
  Metro.joins(:geoboundaries)
       .where("ST_Contains(geoboundaries.boundary, ST_GeomFromText(?, 4326))", "POINT(#{latitude} #{longitude})")
       .first
end