Class: Measures::CqlLoader
- Inherits:
-
Object
- Object
- Measures::CqlLoader
- Defined in:
- lib/measures/loading/cql_loader.rb
Overview
Utility class for loading CQL measure definitions into the database from the MAT export zip
Class Method Summary collapse
-
.composite_measure?(measure_dir) ⇒ Boolean
Returns true if ths uploaded measure zip file is a composite measure.
-
.create_component_measures(current_directory, current_user, measure_details, vsac_options, vsac_ticket_granting_ticket) ⇒ Object
Creates a composite’s component measures.
-
.create_measure(measure_dir, user, measure_details, vsac_options, vsac_ticket_granting_ticket, component_elms = nil) ⇒ Object
Creates and returns a measure.
-
.extract_measures(measure_zip, current_user, measure_details, vsac_options, vsac_ticket_granting_ticket) ⇒ Object
Returns an array of measures Single measure returned into the array if it is a non-composite measure.
-
.generate_single_code_references(elms, all_codes_and_code_names, user) ⇒ Object
Add single code references by finding the codes from the elm and creating new ValueSet objects With a generated GUID as a fake oid.
- .get_files_from_directory(dir) ⇒ Object
-
.get_value_set_oid_version_objects(value_sets, single_code_references) ⇒ Object
Returns a list of objects that include the valueset oids and their versions.
-
.mat_cql_export?(zip_file) ⇒ Boolean
Verifies that the zip file contains a valid measure Works for both regular & composite measures.
-
.modify_value_set_versions(elms) ⇒ Object
Adjusting value set version data.
-
.process_cql(files, main_cql_library, user, vsac_options, vsac_ticket_granting_ticket, measure_id = nil, component_elms = nil) ⇒ Object
Manages all of the CQL processing that is not related to the HQMF.
-
.replace_codesystem_oids_with_names(elms) ⇒ Object
Replace all the code system ids that are oids with the friendly name of the code system TODO: preferred solution would be to continue using OIDs in the ELM and enable Bonnie to supply those OIDs to the calculation engine in patient data and value sets.
-
.retrieve_elm_and_hqmf(files) ⇒ Object
Takes in array of xml files and returns hash with keys HQMF_XML and ELM_XML.
- .set_data_criteria_code_list_ids(json, cql_artifacts) ⇒ Object
-
.unzip_measure_contents(zip_file, tmp_dir) ⇒ Object
Returns the base directory of the measure.
-
.valid_measure_contents?(measure_dir, check_components = false) ⇒ Boolean
Verifies contents of the given measure are valid (works for regular, composite and component measures).
Class Method Details
.composite_measure?(measure_dir) ⇒ Boolean
Returns true if ths uploaded measure zip file is a composite measure
5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 |
# File 'lib/measures/loading/cql_loader.rb', line 5 def self.composite_measure?(measure_dir) # Look through all xml files at current directory level and find QDM files = Dir.glob("#{measure_dir}/**.xml").select begin # Iterate over all files passed in, extract file to temporary directory. files.each do |xml_file| if xml_file && xml_file.size > 0 # Open up xml file and read contents. doc = Nokogiri::XML.parse(File.read(xml_file)) # Check if root node in xml file matches either the HQMF file or ELM file. if doc.root.name == 'QualityMeasureDocument' # Root node for HQMF XML # Xpath to determine if it is a composite or not doc.root.add_namespace_definition('cda', 'urn:hl7-org:v3') return !doc.at_xpath('//cda:measureAttribute[cda:code[@code="MSRTYPE"]][cda:value[@code="COMPOSITE"]]').nil? end end end rescue Exception => e raise MeasureLoadingException.new "Error Checking MAT Export: #{e.}" end false end |
.create_component_measures(current_directory, current_user, measure_details, vsac_options, vsac_ticket_granting_ticket) ⇒ Object
Creates a composite’s component measures
170 171 172 173 174 175 176 177 178 |
# File 'lib/measures/loading/cql_loader.rb', line 170 def self.create_component_measures(current_directory, current_user, measure_details, , vsac_ticket_granting_ticket) component_measures = [] Dir.glob("#{current_directory}/*").sort.each do |file| if File.directory?(file) component_measures << create_measure(file, current_user, measure_details, , vsac_ticket_granting_ticket) end end component_measures end |
.create_measure(measure_dir, user, measure_details, vsac_options, vsac_ticket_granting_ticket, component_elms = nil) ⇒ Object
Creates and returns a measure
181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 |
# File 'lib/measures/loading/cql_loader.rb', line 181 def self.create_measure(measure_dir, user, measure_details, , vsac_ticket_granting_ticket, component_elms=nil) measure = nil # Grabs the cql file contents, the elm_xml contents, elm_json contents and the hqmf file path files = get_files_from_directory(measure_dir) # Load hqmf into HQMF Parser hqmf_model = Measures::Loader.parse_hqmf_model(files[:HQMF_XML_PATH]) # Get main measure from hqmf parser main_cql_library = hqmf_model.cql_measure_library cql_artifacts = process_cql(files, main_cql_library, user, , vsac_ticket_granting_ticket, hqmf_model.hqmf_set_id, component_elms) # Create CQL Measure hqmf_model.backfill_patient_characteristics_with_codes(cql_artifacts[:all_codes_and_code_names]) json = hqmf_model.to_json json.convert_keys_to_strings # Set the code list ids of data criteria and source data criteria that use direct reference codes to GUIDS. json['source_data_criteria'], json['data_criteria'] = set_data_criteria_code_list_ids(json, cql_artifacts) # Create CQL Measure measure_details["composite"] = composite_measure?(measure_dir) measure = Measures::Loader.load_hqmf_cql_model_json(json, user, cql_artifacts[:all_value_set_oids], main_cql_library, cql_artifacts[:cql_definition_dependency_structure], cql_artifacts[:elms], cql_artifacts[:elm_annotations], files[:CQL], measure_details, cql_artifacts[:value_set_oid_version_objects]) measure end |
.extract_measures(measure_zip, current_user, measure_details, vsac_options, vsac_ticket_granting_ticket) ⇒ Object
Returns an array of measures Single measure returned into the array if it is a non-composite measure
120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 |
# File 'lib/measures/loading/cql_loader.rb', line 120 def self.extract_measures(measure_zip, current_user, measure_details, , vsac_ticket_granting_ticket) measure = nil component_measures = [] # Unzip measure contents while retaining the directory structure Dir.mktmpdir do |tmp_dir| current_directory = unzip_measure_contents(measure_zip, tmp_dir) if !valid_measure_contents?(current_directory, true) raise MeasureLoadingException.new("Zip file was not a MAT package.") end component_elms = {} component_elms[:ELM_JSON] = [] # If it is a composite measure, load in each of the components # Components must be loaded first so their elms can be passed onto the composite if composite_measure?(current_directory) component_measures = create_component_measures(current_directory, current_user, measure_details, , vsac_ticket_granting_ticket) component_measures.each do |component_measure| component_elms[:ELM_JSON].push(*component_measure.elm) end end # Load in regular/composite measure measure begin measure = create_measure(current_directory, current_user, measure_details, , vsac_ticket_granting_ticket, component_elms) rescue => e component_measures.each { |component| component.delete } raise e end # Create, associate and save the measure package. measure_package = CqlMeasurePackage.new(file: BSON::Binary.new(measure_zip.read())) measure.package = measure_package measure.package.save component_measures.each do |component_measure| # Update the components' hqmf_set_id, formatted as follows: # <composite_hqmf_set_id>&<component_hqmf_set_id> component_measure.hqmf_set_id = measure.hqmf_set_id + '&' + component_measure.hqmf_set_id component_measure.component = true; # Associate the component with the composite measure.component_hqmf_set_ids.push(component_measure.hqmf_set_id) end end # End of temporary directory usage # Put measure (and component measures) into an array to return measures = component_measures << measure return measures end |
.generate_single_code_references(elms, all_codes_and_code_names, user) ⇒ Object
Add single code references by finding the codes from the elm and creating new ValueSet objects With a generated GUID as a fake oid.
412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 |
# File 'lib/measures/loading/cql_loader.rb', line 412 def self.generate_single_code_references(elms, all_codes_and_code_names, user) single_code_references = [] # Add all single code references from each elm file elms.each do | elm | # Check if elm has single reference code. if elm['library'] && elm['library']['codes'] && elm['library']['codes']['def'] # Loops over all single codes and saves them as fake valuesets. elm['library']['codes']['def'].each do |code_reference| code_sets = {} # look up the referenced code system code_system_def = elm['library']['codeSystems']['def'].find { |code_sys| code_sys['name'] == code_reference['codeSystem']['name'] } code_system_name = code_system_def['id'] code_system_version = code_system_def['version'] code_sets[code_system_name] ||= [] code_sets[code_system_name] << code_reference['id'] # Generate a unique number as our fake "oid" based on parameters that identify the DRC code_hash = "drc-" + Digest::SHA2.hexdigest("#{code_system_name} #{code_reference['id']} #{code_reference['name']} #{code_system_version}") # Keep a list of generated_guids and a hash of guids with code system names and codes. single_code_references << { guid: code_hash, code_system_name: code_system_name, code: code_reference['id'] } all_codes_and_code_names[code_hash] = code_sets # code_hashs are unique hashes, there's no sense in adding duplicates to the ValueSet collection if !HealthDataStandards::SVS::ValueSet.all().where(oid: code_hash, user_id: user.id).first() # Create a new "ValueSet" and "Concept" object and save. valueSet = HealthDataStandards::SVS::ValueSet.new({oid: code_hash, display_name: code_reference['name'], version: '' ,concepts: [], user_id: user.id}) concept = HealthDataStandards::SVS::Concept.new({code: code_reference['id'], code_system_name: code_system_name, code_system_version: code_system_version, display_name: code_reference['name']}) valueSet.concepts << concept valueSet.save! end end end end # Returns a list of single code objects and a complete list of code systems and codes for all valuesets on the measure. return single_code_references, all_codes_and_code_names end |
.get_files_from_directory(dir) ⇒ Object
210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 |
# File 'lib/measures/loading/cql_loader.rb', line 210 def self.get_files_from_directory(dir) cql_paths = Dir.glob(File.join("#{dir}/**.cql")).sort xml_paths = Dir.glob(File.join("#{dir}/**.xml")).sort elm_json_paths = Dir.glob(File.join("#{dir}/**.json")).sort begin cql_contents = [] cql_paths.each do |cql_path| cql_contents << open(cql_path).read end elm_json = [] elm_json_paths.each do |elm_json_path| elm_json << open(elm_json_path).read end xml_file_hash = retrieve_elm_and_hqmf(xml_paths) elm_xml_paths = xml_file_hash[:ELM_XML] elm_xml = [] elm_xml_paths.each do |elm_xml_path| elm_xml << open(elm_xml_path).read end files = { :HQMF_XML_PATH => xml_file_hash[:HQMF_XML], :ELM_JSON => elm_json, :CQL => cql_contents, :ELM_XML => elm_xml } return files rescue Exception => e raise MeasureLoadingException.new "Error Parsing Measure Logic: #{e.}" end end |
.get_value_set_oid_version_objects(value_sets, single_code_references) ⇒ Object
Returns a list of objects that include the valueset oids and their versions
359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 |
# File 'lib/measures/loading/cql_loader.rb', line 359 def self.get_value_set_oid_version_objects(value_sets, single_code_references) # [LDC] need to make this an array of objects instead of a hash because Mongo is # dumb and *let's you* have dots in keys on object creation but *doesn't let you* # have dots in keys on object update or retrieve.... value_set_oid_version_objects = [] value_sets.each do |vs| value_set_oid_version_objects << {:oid => vs.oid, :version => vs.version} end single_code_references.each do |single_code| # Only add unique Direct Reference Codes to the object unless value_set_oid_version_objects.include?({:oid => single_code[:guid], :version => ""}) value_set_oid_version_objects << {:oid => single_code[:guid], :version => ""} end end # Return a list of unique objects only value_set_oid_version_objects end |
.mat_cql_export?(zip_file) ⇒ Boolean
Verifies that the zip file contains a valid measure Works for both regular & composite measures
30 31 32 33 34 35 36 37 38 |
# File 'lib/measures/loading/cql_loader.rb', line 30 def self.mat_cql_export?(zip_file) # Extract contents of zip file while retaining the directory structure original = Dir.pwd Dir.mktmpdir do |tmp_dir| current_directory = unzip_measure_contents(zip_file, tmp_dir) # Check if measure contents are valid return valid_measure_contents?(current_directory, true) end end |
.modify_value_set_versions(elms) ⇒ Object
Adjusting value set version data. If version is profile, set the version to nil
394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 |
# File 'lib/measures/loading/cql_loader.rb', line 394 def self.modify_value_set_versions(elms) elms.each do |elm| if elm['library']['valueSets'] && elm['library']['valueSets']['def'] elm['library']['valueSets']['def'].each do |value_set| # If value set has a version and it starts with 'urn:hl7:profile:' then set to nil if value_set['version'] && value_set['version'].include?('urn:hl7:profile:') value_set['profile'] = URI.decode(value_set['version'].split('urn:hl7:profile:').last) value_set['version'] = nil # If value has a version and it starts with 'urn:hl7:version:' then strip that and keep the actual version value. elsif value_set['version'] && value_set['version'].include?('urn:hl7:version:') value_set['version'] = URI.decode(value_set['version'].split('urn:hl7:version:').last) end end end end end |
.process_cql(files, main_cql_library, user, vsac_options, vsac_ticket_granting_ticket, measure_id = nil, component_elms = nil) ⇒ Object
Manages all of the CQL processing that is not related to the HQMF.
267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 |
# File 'lib/measures/loading/cql_loader.rb', line 267 def self.process_cql(files, main_cql_library, user, , vsac_ticket_granting_ticket, measure_id=nil, component_elms=nil) elm_strings = files[:ELM_JSON] # Removes 'urn:oid:' from ELM for Bonnie and Parse the JSON elm_strings.each { |elm_string| elm_string.gsub! 'urn:oid:', '' } elms = elm_strings.map{ |elm| JSON.parse(elm, :max_nesting=>1000)} elm_annotations = parse_elm_annotations(files[:ELM_XML]) if (!component_elms.nil?) elms.push(*component_elms[:ELM_JSON]) end # Hash of define statements to which define statements they use. cql_definition_dependency_structure = populate_cql_definition_dependency_structure(main_cql_library, elms) begin # Go back for the library statements cql_definition_dependency_structure = populate_used_library_dependencies(cql_definition_dependency_structure, main_cql_library, elms) # Add unused libraries to structure and set the value to empty hash cql_definition_dependency_structure = populate_unused_included_libraries(cql_definition_dependency_structure, elms) rescue => e raise MeasureLoadingException.new("Measure package missing a library or component.") end # fix up statement names in cql_statement_dependencies to not use periods <<WRAP 1>> # this is matched with an UNWRAP in MeasuresController in the bonnie project Measures::MongoHashKeyWrapper::wrapKeys cql_definition_dependency_structure # Depening on the value of the value set version, change it to null, strip out a substring or leave it alone. modify_value_set_versions(elms) # Grab the value sets from the elm elm_value_sets = [] elms.each do | elm | # Confirm the library has value sets if elm['library'] && elm['library']['valueSets'] && elm['library']['valueSets']['def'] elm['library']['valueSets']['def'].each do |value_set| elm_value_sets << {oid: value_set['id'], version: value_set['version'], profile: value_set['profile']} end end end # Get Value Sets value_set_models = [] # Only load value sets from VSAC if there is a ticket_granting_ticket. if !vsac_ticket_granting_ticket.nil? value_set_models = Measures::ValueSetLoader.load_value_sets_from_vsac(elm_value_sets, , vsac_ticket_granting_ticket, user, measure_id) else # No vsac credentials were provided grab the valueset and valueset versions from the 'value_set_oid_version_object' on the existing measure db_measure = CqlMeasure.by_user(user).where(hqmf_set_id: measure_id).first unless db_measure.nil? measure_value_set_version_map = db_measure.value_set_oid_version_objects measure_value_set_version_map.each do |value_set| query_params = {user_id: user.id, oid: value_set['oid'], version: value_set['version']} value_set = HealthDataStandards::SVS::ValueSet.where(query_params).first() if value_set value_set_models << value_set else raise MeasureLoadingException.new "Value Set not found in database: #{query_params}" end end end end # Get code systems and codes for all value sets in the elm. all_codes_and_code_names = HQMF2JS::Generator::CodesToJson.from_value_sets(value_set_models) # Replace code system oids with friendly names # TODO: preferred solution would be to continue using OIDs in the ELM and enable Bonnie to supply those OIDs # to the calculation engine in patient data and value sets. replace_codesystem_oids_with_names(elms) # Generate single reference code objects and a complete list of code systems and codes for the measure. single_code_references, all_codes_and_code_names = generate_single_code_references(elms, all_codes_and_code_names, user) # Add our new fake oids to measure value sets. all_value_set_oids = value_set_models.collect{|vs| vs.oid} single_code_references.each do |single_code| # Only add unique Direct Reference Codes unless all_value_set_oids.include?(single_code[:guid]) all_value_set_oids << single_code[:guid] end end # Add a list of value set oids and their versions value_set_oid_version_objects = get_value_set_oid_version_objects(value_set_models, single_code_references) cql_artifacts = {:elms => elms, :elm_annotations => elm_annotations, :cql_definition_dependency_structure => cql_definition_dependency_structure, :all_value_set_oids => all_value_set_oids, :value_set_oid_version_objects => value_set_oid_version_objects, :single_code_references => single_code_references, :all_codes_and_code_names => all_codes_and_code_names} end |
.replace_codesystem_oids_with_names(elms) ⇒ Object
Replace all the code system ids that are oids with the friendly name of the code system TODO: preferred solution would be to continue using OIDs in the ELM and enable Bonnie to supply those OIDs
to the calculation engine in patient data and value sets.
380 381 382 383 384 385 386 387 388 389 390 391 |
# File 'lib/measures/loading/cql_loader.rb', line 380 def self.replace_codesystem_oids_with_names(elms) elms.each do |elm| # Only do replacement if there are any code systems in this library. if elm['library'].has_key?('codeSystems') elm['library']['codeSystems']['def'].each do |code_system| code_name = HealthDataStandards::Util::CodeSystemHelper.code_system_for(code_system['id']) # if the helper returns "Unknown" then keep what was there code_system['id'] = code_name unless code_name == "Unknown" end end end end |
.retrieve_elm_and_hqmf(files) ⇒ Object
Takes in array of xml files and returns hash with keys HQMF_XML and ELM_XML
244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 |
# File 'lib/measures/loading/cql_loader.rb', line 244 def self.retrieve_elm_and_hqmf(files) file_paths_hash = {} file_paths_hash[:ELM_XML] = [] begin files.each do |xml_file_path| if xml_file_path && xml_file_path.size > 0 # Open up xml file and read contents. doc = Nokogiri::XML.parse(File.read(xml_file_path)) # Check if root node in xml file matches either the HQMF file or ELM file. if doc.root.name == 'QualityMeasureDocument' # Root node for HQMF XML file_paths_hash[:HQMF_XML] = xml_file_path elsif doc.root.name == 'library' # Root node for ELM XML file_paths_hash[:ELM_XML] << xml_file_path end end end rescue Exception => e raise MeasureLoadingException.new "Error Checking MAT Export: #{e.}" end file_paths_hash end |
.set_data_criteria_code_list_ids(json, cql_artifacts) ⇒ Object
93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 |
# File 'lib/measures/loading/cql_loader.rb', line 93 def self.set_data_criteria_code_list_ids(json, cql_artifacts) # Loop over data criteria to search for data criteria that is using a single reference code. # Once found set the Data Criteria's 'code_list_id' to our fake oid. Do the same for source data criteria. json['data_criteria'].each do |data_criteria_name, data_criteria| unless data_criteria['code_list_id'] if data_criteria['inline_code_list'] # Check to see if inline_code_list contains the correct code_system and code for a direct reference code. data_criteria['inline_code_list'].each do |code_system, code_list| # Loop over all single code reference objects. cql_artifacts[:single_code_references].each do |single_code_object| # If Data Criteria contains a matching code system, check if the correct code exists in the data critera values. # If both values match, set the Data Criteria's 'code_list_id' to the single_code_object_guid. if code_system == single_code_object[:code_system_name] && code_list.include?(single_code_object[:code]) data_criteria['code_list_id'] = single_code_object[:guid] # Modify the matching source data criteria json['source_data_criteria'][data_criteria_name + "_source"]['code_list_id'] = single_code_object[:guid] end end end end end end return json['source_data_criteria'], json['data_criteria'] end |
.unzip_measure_contents(zip_file, tmp_dir) ⇒ Object
Returns the base directory of the measure
41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 |
# File 'lib/measures/loading/cql_loader.rb', line 41 def self.unzip_measure_contents(zip_file, tmp_dir) Zip::ZipFile.open(zip_file.path) do |zip_file| zip_file.each do |f| f_path = File.join(tmp_dir, f.name) FileUtils.mkdir_p(File.dirname(f_path)) f.extract(f_path) end end current_directory = tmp_dir # Detect if the zip file contents were stored into a single directory if Dir.glob("#{current_directory}/*").count < 3 # If there is a single folder containing the zip file contents, step into it (ignore __MACOSX file if it exists) Dir.glob("#{current_directory}/*").select.each do |file| if !file.end_with?('__MACOSX') && File.directory?(file) current_directory = file break end end end return current_directory end |
.valid_measure_contents?(measure_dir, check_components = false) ⇒ Boolean
Verifies contents of the given measure are valid (works for regular, composite and component measures)
64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 |
# File 'lib/measures/loading/cql_loader.rb', line 64 def self.valid_measure_contents?(measure_dir, check_components = false) # If composite measure given, check components if check_components Dir.glob("#{measure_dir}/*").each do |file| if File.directory?(file) if !valid_measure_contents?(file) return false end end end end # Grab all cql, elm & human readable docs from measure cql_entry = Dir.glob(File.join(measure_dir,'**.cql')).select {|x| !File.basename(x).starts_with?('__MACOSX') }.first elm_json = Dir.glob(File.join(measure_dir,'**.json')).select {|x| !File.basename(x).starts_with?('__MACOSX') }.first human_readable_entry = Dir.glob(File.join(measure_dir,'**.html')).select {|x| !File.basename(x).starts_with?('__MACOSX') }.first # Grab all xml files in the measure. xml_files = Dir.glob(File.join(measure_dir,'**.xml')).select {|x| !File.basename(x).starts_with?('__MACOSX') } # Find key value pair for HQMF and ELM xml files. if xml_files.count > 0 xml_files_hash = retrieve_elm_and_hqmf(xml_files) !cql_entry.nil? && !elm_json.nil? && !human_readable_entry.nil? && !xml_files_hash[:HQMF_XML].nil? && !xml_files_hash[:ELM_XML].nil? else false end end |