AIPP
Parser for Aeronautical Information Publication (AIP) available online.
This gem incluces two executables to download and parse aeronautical data as HTML, PDF, XSLX, ODS and CSV, then build and export is as AIXM or OFMX.
- Homepage
- Rubydoc
- Author: Sven Schwyn - Bitcetera
Table of Contents
Install
Usage
Storage
Regions
AIRAC Date Calculations
References
Development
Install
Security
This gem is cryptographically signed in order to assure it hasn't been tampered with. Unless already done, please add the author's public key as a trusted certificate now:
gem cert --add <(curl -Ls https://raw.github.com/svoop/aipp/main/certs/svoop.pem)
Standalone
Make sure to have the latest version of Ruby and then install this gem:
gem install aipp --trust-policy MediumSecurity
Bundler
If you're familiar with Bundler powered Ruby projects, you might prefer to add the following to your Gemfile or gems.rb:
gem aipp
And then install the bundle:
bundle install --trust-policy MediumSecurity
Usage
See the built-in help for all options:
aip2aixm --help
aip2ofmx --help
Say, you with to build the complete OFMX file for the current AIRAC cycle of the region LF:
aip2ofmx -r LF
You'll find the OFMX file in the current directory if the binary exits successfully.
Storage
AIPP uses a storage directory for configuration, caching and in order to keep the results of previous runs. The default location is ~/.aipp
, however, you can pass a different directory with the --storage
argument.
You'll find a directory for each region which contains the following items:
sources/
This directory contains one ZIP archive per AIRAC cycle which incrementially caches all source files used to build the AIXM/OFMX file. Therefore, to make sure it contains all source files for a region, you have to build at least one complete AIXM/OFMX file for that region.builds/
This directory contains one ZIP archive per AIRAC cycle which is overwritten on every run. Therefore, to make sure it contains the complete build for a region, you have to make sure that your last run builds the complete AIXM/OFMX for that region. This archive contains:- the built AIXM/OFMX file
build.yaml
– context of the build processmanifest.csv
– diffable manifest (see below)
config.yml
This file contains configuration which will be read on subsequent runs, most notably the namespace UUID used to identify the creator of OFMX files.
The manifest is a CSV which lists every feature on a separate line along with its hashes, AIP and comment. You can diff
or git diff
two manifests:
$ git diff -U0 2019-09-12/manifest.csv 2019-10-10/manifest.csv
--- a/2019-09-12/manifest.csv
+++ b/2019-10-10/manifest.csv
@@ -204 +204 @@ AD-1.3,Ahp,9e9f031e,d6f22057,Airport: LFLJ COURCHEVEL
-AD-1.3,Ahp,9f1eed18,37ddbbde,Airport: LFQD ARRAS ROCLINCOURT
+AD-1.3,Ahp,9f1eed18,f0e60105,Airport: LFQD ARRAS ROCLINCOURT
@@ -312 +312 @@ AD-2,Aha,4250c9ee,04d49dc7,Address: RADIO for LFHV
-AD-2,Aha,6b381b32,fb947716,Address: RADIO for LFPO
+AD-2,Aha,6b381b32,b9723b7e,Address: RADIO for LFPO
@@ -664 +663,0 @@ AD-2,Ser,3920a7fd,4545c5eb,Service: AFIS by LFGA TWR
-AD-2,Ser,39215774,1f13f2cf,Service: APP by LFCR APP
@@ -878 +876,0 @@ AD-2,Ser,bb5228d7,7cfb4572,Service: TWR by LFMH TWR
-AD-2,Ser,bc72caf2,0a15b39c,Service: FIS by LFCR FIC
(...)
The advantage of git diff
is it's ability to hightlight exactly which part of a line has changed. Check out this post to learn how.
Regions
The reference implementation is region "LF" (France).
To implement a region, you have to create a new directory lib/aipp/regions/{REGION}/ and place the following files there:
AIP Parsers
Say, you want to parse ENR-4.3, you have to create the file ENR-4.3.rb which defines the class AIPP::LF::ENR43
as follows:
module AIPP
module LF
class ENR43 < AIP
DEPENDS = %w(ENR-2.1 ENR-2.2) # declare dependencies to other AIPs
end
end
end
The class has to implement some methods either in the class itself or in a helper included by the class.
⚠️ Parser files usually follow AIP naming conventions such as ENR-4.3
. However, you're free to use arbitrary naming for parser files e.g. if you're working with one big data source which contains the full AIP dataset.
Mandatory parse
Method
The class must implement the parse
method which contains the code to read, parse and write the data:
module AIPP
module LF
class ENR43 < AIP
def parse
html = read # read the Nokogiri::HTML5 document
feature = (...) # build the feature
add(feature: feature) # add the feature to AIXM::Document
end
end
end
end
Some AIP may be split over several files which require a little more code to load the individual HTML source files:
module AIPP
module LF
class AD2 < AIP
def parse
%i(one two three).each do |part|
html = read("#{aip}.#{part}") # read with a non-standard name
support_html = read('AD-0.6') # maybe read necessary support documents
(...)
end
end
end
end
end
Inside the parse
method, you have access to the following methods:
Method | Description |
---|---|
read |
download and read an AIP file |
add |
add a AIXM::Feature |
find |
find previously written AIXM::Feature s by object |
find_by |
find previously written AIXM::Feature s by class and attribute values |
unique |
prevent duplicate AIXM::Feature s |
given |
inline condition for assignments |
link_to |
optionally checked Markdown link |
Object#blank and String |
some core extensions from ActiveSupport |
aip |
AIP name (equal to the parser file name without its file extension such as "ENR-2.1" implemented in the file "ENR-2.1.rb") |
aip_file |
AIP file as passed and possibly renamed by url_for |
options |
arguments read from aip2aixm or aip2ofmx respectively |
config |
configuration read from config.yml |
borders |
borders defined as GeoJSON read from the region (see below) |
cache |
OStruct instance to make objects available across AIPs |
To make the parser code more readable, this gem provides a few useful core extensions as well:
Mandatory url_for
Method
The class must implement the url_for
method which returns the URL from where to download the AIP file:
module AIPP
module LF
class AD2 < AIP
def url_for(aip_file)
# build and return the download URL for the aip file
end
end
end
end
There are a few things to note about url_for
:
- If the returned string begins with a protocol like
https:
, the downloader will fetch the file from there. - If the returned string is just a file name, the downloader will look for this exact file in the current local directory.
- The passed
aip_file
will be used as the file name for the local copy in the sources directory. You can rename it on the fly by assigning a new value to this variable.
Optional setup
Method
The class may implement the setup
method. If present, it will be called when this parser is instantiated:
module AIPP
module LF
class AD2 < AIP
def setup
AIXM.config.voice_channel_separation = :any
cache.setup_at ||= Time.now
end
end
end
end
Borders
AIXM knows named borders for country boundaries. However, you might need additional borders which don't exist as named borders.
You can define additional borders as AIPP::Border
objects in two ways.
From GeoJSON
Create simple GeoJSON files in the lib/aipp/regions/{REGION}/borders/ directory, for example this my_border_1.geojson
:
{
"type": "GeometryCollection",
"geometries": [
{
"type": "LineString",
"coordinates": [
[6.009531650000042, 45.12013319700009],
[6.015747738000073, 45.12006702600007]
]
},
{
"type": "LineString",
"coordinates": [
[4.896732957592112, 43.95662950764992],
[4.005739165537195, 44.10769266295027]
]
}
]
}
⚠️ The GeoJSON file must consist of exactly one GeometryCollection
which may contain any number of LineString
geometries. Only LineString
geometries are recognized! To define a closed polygon, the first coordinates of a LineString
must be identical to the last coordinates.
From Coordinates
It's also possible to create a AIPP::Border
objects on the fly:
my_border_2 = AIPP::Border.from_array(
[
["6.009531650000042 45.12013319700009", "6.015747738000073 45.12006702600007"],
["4.896732957592112 43.95662950764992", "4.005739165537195 44.10769266295027"]
]
)
The coordinate pairs must be separated with whitespaces and/or commas. If you want to use this border everywhere, make sure you add it to the others:
borders["my_border_2"] = my_border_2
Usage in Parsers
In the parser, the borders
method gives you access to all borders read from GeoJSON files:
borders # => { "my_border_1" => #<AIPP::Border>, "my_border_2" => #<AIPP::Border> }
The border object implements simple nearest point and segment calculations to create arrays of AIXM::XY
which can be used with AIXM::Component::Geometry
.
See AIPP::Border
for more on this.
Helpers
Helpers are modules defined in the lib/aipp/regions/{REGION}/helpers/ directory. All helper modules are required automatically in alphabetic order.
Fixtures and Patches
Fixtures are static YAML data files in the lib/aipp/regions/{REGION}/fixtures/ directory. All fixtures are read automatically. Please note that the name of the AIP parser (e.g. AD-1.3.rb
) must match the name of the corresponding fixture (e.g. fixtures/AD-1.3.yml
).
When parsed data is faulty or missing, you may fall back to such static data instead. This is where patches come in. You can patch any AIXM attribute setter by defining a patch block inside the AIP parser and accessing the static data via parser.fixture
:
module AIPP
module LF
class AD2 < AIP
patch AIXM::Feature::Airport, :z do |parser, object, value|
throw(:abort) unless value.nil?
throw(:abort, 'fixture missing') unless z = parser.fixture.dig(object.id, 'z')
AIXM.z(z, :qnh)
end
end
end
end
The patch receives the object and the value which is about to be assigned. It should implement something along these lines:
- If the value is okay,
throw(:abort)
to leave the patch block without touching anything. - Otherwise, try to fetch a better value e.g. from the fixtures. If no better value can be found (e.g. outdated fixtures),
throw(:abort, "reason")
to leave the patch block and fail with a useful error message which contains the reason thrown. - At last, build and return the value object which will be assigned instead of the original value.
In case the object
does not carry enough details, you can access instance variables of the parser like so:
parser.instance_variable_get(:@instance_variable)
Source File Line Numbers
In order to reference the source of an AIXM/OFMX feature, it's necessary to know the line number where a particular node occurs in the HTML source file. You can ask any HTML element as follows:
tr.line
Errors
You should fail
on fatal problems which must be fixed. The --debug-on-error
command line argument will open a debug session when such an error occurs. Issue errors as usual:
fail "my message"
Warnings
You should warn
on non-fatal problems which should be fixed (default) or might be ignored (severe: false
). The --debug-on-warning ID
command line argument will open a debug session when then warning with the given ID occurs. To issue a warning:
warn("my message")
warn("my message", severe: false) # show warning only when --unsevere-warn argument is set
Messages
info
Use info
for essential info messages:
info("my message") # displays "my message" in black
info("my message", color: :green) # displays "my message" in green
verbose info
Use verbose_info
for in-depth info messages which are only shown if the --verbose
mode is set:
verbose_info("my message") # displays "my message" in blue
debug
The default Ruby debugger is enabled by default, you can add a breakpoint as usual with:
debugger
AIRAC Date Calculations
The AIPP::AIRAC
class is used to calculate AIRAC cycles:
airac = AIPP::AIRAC.new(Date.parse('2017-12-24'))
airac.date # => 2018-12-07
airac.id # => 1713
airac.next_date # => 2018-01-04
airac.next_id # => 1801
References
- Geo Maps – programmatically generated GeoJSON maps
- open flightmaps – open-source aeronautical maps
- AIXM Rubygem – AIXM/OFMX generator for Ruby
Development
To install the development dependencies and then run the test suite:
bundle install
bundle exec rake # run tests once
bundle exec guard # run tests whenever files are modified
Please submit issues on:
https://github.com/svoop/aipp/issues
To contribute code, fork the project on Github, add your code and submit a pull request:
https://help.github.com/articles/fork-a-repo
License
The gem is available as open source under the terms of the MIT License.