Class: Queries::Query::Filter

Inherits:
Queries::Query show all
Includes:
Concerns::Users
Defined in:
lib/queries/query/filter.rb

Overview

Overview

This class manages params and nesting of filter queries.

Each inheriting class defines a PARAMS variable. Each concern defines ‘self.params`. Together these two lists are used to compose a list of acceptable params, dynamically, based on the nature of the nested queries.

Test coverage is currently in /spec/lib/queries/otu/filter_spec.rb.

!! When adding a new query tests do some linting of parameters, constants etc. Run them early and often !!

Constant Summary collapse

SUBQUERIES =

!! SUBQUERIES is cross-referenced in app/views/javascript/vue/components/radials/filter/links/*.js models. !! When you add a reference here, ensure corresponding js model is aligned. There are tests that will catch if they are not.

For example: github.com/SpeciesFileGroup/taxonworks/app/javascript/vue/components/radials/filter/constants/queryParam.js github.com/SpeciesFileGroup/taxonworks/app/javascript/vue/components/radials/filter/constants/filterLinks.js github.com/SpeciesFileGroup/taxonworks/app/javascript/vue/components/radials/filter/links/CollectionObject.js

You may also need a reference in app/javascript/vue/routes/routes.js app/javascript/vue/components/radials/linker/links

This is read as :to <- [:from1, from2…] ].

{
  asserted_distribution: [:source, :otu, :biological_association, :taxon_name],
  biological_association: [:source, :collecting_event, :otu, :collection_object, :taxon_name, :asserted_distribution],
  biological_associations_graph: [:biological_association, :source],
  collecting_event: [:source, :collection_object, :biological_association, :otu, :image, :taxon_name],
  collection_object: [:source, :loan, :otu, :taxon_name, :collecting_event, :biological_association, :extract, :image, :observation],
  content: [:source, :otu, :taxon_name, :image],
  controlled_vocabulary_term: [:data_attribute],
  data_attribute: [:collection_object, :collecting_event, :taxon_name],
  dwc_occurrence: [:asserted_distribution, :collection_object],
  descriptor: [:source, :observation, :otu],
  extract: [:source, :otu, :collection_object, :observation],
  image: [:content, :collection_object, :collecting_event, :otu, :observation, :source, :taxon_name ],
  loan: [:collection_object, :otu],
  observation: [:collection_object, :descriptor, :image, :otu, :source, :taxon_name],
  otu: [:asserted_distribution, :biological_association, :collection_object, :collecting_event, :content, :descriptor, :extract, :image, :loan, :observation, :source, :taxon_name ],
  person: [],
  source: [:asserted_distribution,  :biological_association, :collecting_event, :collection_object, :content, :descriptor, :extract, :image, :observation, :otu, :taxon_name],
  taxon_name: [:asserted_distribution, :biological_association, :collection_object, :collecting_event, :image, :otu, :source ]
}.freeze
FILTER_QUERIES =

We could consider ‘.safe_constantize` to make this a f(n), but we’d have to have a list somewhere else anyways to further restrict allowed classes.

{
  asserted_distribution_query: '::Queries::AssertedDistribution::Filter',
  biological_association_query: '::Queries::BiologicalAssociation::Filter',
  biological_associations_graph_query: '::Queries::BiologicalAssociationsGraph::Filter',
  collecting_event_query: '::Queries::CollectingEvent::Filter',
  collection_object_query: '::Queries::CollectionObject::Filter',
  content_query: '::Queries::Content::Filter',
  controlled_vocabulary_term_query: '::Queries::ControlledVocabularyTerm::Filter',
  data_attribute_query: '::Queries::DataAttribute::Filter',
  descriptor_query: '::Queries::Descriptor::Filter',
  dwc_occurrence_query: '::Queries::DwcOccurrence::Filter',
  extract_query: '::Queries::Extract::Filter',
  image_query: '::Queries::Image::Filter',
  loan_query: '::Queries::Loan::Filter',
  observation_query: '::Queries::Observation::Filter',
  otu_query: '::Queries::Otu::Filter',
  person_query: '::Queries::Person::Filter',
  source_query: '::Queries::Source::Filter',
  taxon_name_query: '::Queries::TaxonName::Filter',
}.freeze

Instance Attribute Summary collapse

Attributes inherited from Queries::Query

#query_string, #terms

Class Method Summary collapse

Instance Method Summary collapse

Methods inherited from Queries::Query

#alphabetic_strings, #alphanumeric_strings, base_name, #base_name, #base_query, #build_terms, #cached_facet, #end_wildcard, #levenshtein_distance, #match_ordered_wildcard_pieces_in_cached, #no_terms?, #referenced_klass, referenced_klass, #referenced_klass_except, #referenced_klass_intersection, #referenced_klass_union, #start_and_end_wildcard, #start_wildcard, #table, #wildcard_pieces

Constructor Details

#initialize(query_params) ⇒ Object

Returns Hash.



195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
# File 'lib/queries/query/filter.rb', line 195

def initialize(query_params)

  # Reference to query_params, i.e. always permitted
  @api = boolean_param(query_params, :api)
  @recent = boolean_param(query_params, :recent)
  @object_global_id = query_params[:object_global_id]

  # !! This is the *only* place Current.project_id should be seen !! It's still not the best
  # way to implement this, but we use it to optimize the scope of sub/nested-queries efficiently.
  # Ideally we'd have a global class param that stores this that all Filters would have access to,
  # rather than an instance variable.
  @project_id = query_params[:project_id] || Current.project_id

  @paginate = boolean_param(query_params, :paginate)
  @per = query_params[:per]
  @page = query_params[:page]

  # After this point, if you started with ActionController::Parameters,
  # then all values have been explicitly permitted.
  if query_params.kind_of?(Hash)
    @params = query_params
  elsif query_params.kind_of?(ActionController::Parameters)
    @params = deep_permit(query_params).to_hash.deep_symbolize_keys
  elsif query_params.nil?
    @params = {}
  else
    raise TaxonWorks::Error, "can not initialize filter with #{query_params.class.name}"
  end

  set_identifier_params(params)
  set_nested_queries(params)
  set_user_dates(params)
end

Instance Attribute Details

#apiObject

Returns Boolean When true api_except_params is applied.

Returns:

  • Boolean When true api_except_params is applied



185
186
187
# File 'lib/queries/query/filter.rb', line 185

def api
  @api
end

#asserted_distribution_queryQuery::AssertedDistributionn::Filter?

Returns:

  • (Query::AssertedDistributionn::Filter, nil)


132
133
134
# File 'lib/queries/query/filter.rb', line 132

def asserted_distribution_query
  @asserted_distribution_query
end

#biological_association_queryQuery::BiologicalAssociation::Filter?

Returns:

  • (Query::BiologicalAssociation::Filter, nil)


135
136
137
# File 'lib/queries/query/filter.rb', line 135

def biological_association_query
  @biological_association_query
end

#biological_associations_graph_queryQuery::BiologicalAssociationsGraph::Filter?

Returns:

  • (Query::BiologicalAssociationsGraph::Filter, nil)


138
139
140
# File 'lib/queries/query/filter.rb', line 138

def biological_associations_graph_query
  @biological_associations_graph_query
end

#collecting_event_queryQuery::CollectingEvent::Filter?

Returns:

  • (Query::CollectingEvent::Filter, nil)


147
148
149
# File 'lib/queries/query/filter.rb', line 147

def collecting_event_query
  @collecting_event_query
end

#collection_object_queryQuery::TaxonName::Filter?

Returns:

  • (Query::TaxonName::Filter, nil)


144
145
146
# File 'lib/queries/query/filter.rb', line 144

def collection_object_query
  @collection_object_query
end

#content_queryQuery::Content::Filter?

Returns:

  • (Query::Content::Filter, nil)


150
151
152
# File 'lib/queries/query/filter.rb', line 150

def content_query
  @content_query
end

#controlled_vocabulary_term_queryQuery::ControlledVocabularyTerm::Filter?

Returns:

  • (Query::ControlledVocabularyTerm::Filter, nil)


141
142
143
# File 'lib/queries/query/filter.rb', line 141

def controlled_vocabulary_term_query
  @controlled_vocabulary_term_query
end

#data_attribute_queryQuery::DataAttribute::Filter?

Returns:

  • (Query::DataAttribute::Filter, nil)


153
154
155
# File 'lib/queries/query/filter.rb', line 153

def data_attribute_query
  @data_attribute_query
end

#descriptor_queryQuery::Descriptor::Filter?

Returns:

  • (Query::Descriptor::Filter, nil)


156
157
158
# File 'lib/queries/query/filter.rb', line 156

def descriptor_query
  @descriptor_query
end

#extract_queryQuery::Extract::Filter?

Returns:

  • (Query::Extract::Filter, nil)


168
169
170
# File 'lib/queries/query/filter.rb', line 168

def extract_query
  @extract_query
end

#image_queryQuery::Image::Filter?

Returns:

  • (Query::Image::Filter, nil)


159
160
161
# File 'lib/queries/query/filter.rb', line 159

def image_query
  @image_query
end

#loan_queryQuery::Loan::Filter?

Returns:

  • (Query::Loan::Filter, nil)


174
175
176
# File 'lib/queries/query/filter.rb', line 174

def loan_query
  @loan_query
end

#object_global_idArray

Locally these look like gid://taxon-works/Otu/1 Using a global id is equivalent to using <model>_id. I.e. it simply restricts the filter to those matching Model#id.

!! If any global id model name does not match the current filter, then then facet is completely rejected.

Returns:

  • (Array)


127
128
129
# File 'lib/queries/query/filter.rb', line 127

def object_global_id
  @object_global_id
end

#observation_queryQuery::Observation::Filter?

Returns:

  • (Query::Observation::Filter, nil)


171
172
173
# File 'lib/queries/query/filter.rb', line 171

def observation_query
  @observation_query
end

#otu_queryQuery::Otu::Filter?

Returns:

  • (Query::Otu::Filter, nil)


165
166
167
# File 'lib/queries/query/filter.rb', line 165

def otu_query
  @otu_query
end

#pageObject

Returns Integer, nil required if paginate == true.

Returns:

  • Integer, nil required if paginate == true



109
110
111
# File 'lib/queries/query/filter.rb', line 109

def page
  @page
end

#paginateObject

Apply pagination within Filter scope

true - apply per and page
false, nil - ignored


105
106
107
# File 'lib/queries/query/filter.rb', line 105

def paginate
  @paginate
end

#paramsObject (readonly)

!! Using setters directly on query parameters will not alter this variable !! !! This is used strictly during the permission process of ActionController::Parameters !!

Returns:

  • Hash the parsed/permitted params

    that were used to on initialize() only!!
    


192
193
194
# File 'lib/queries/query/filter.rb', line 192

def params
  @params
end

#perObject

Returns Integer, nil paginate must equal true page must be !nil?.

Returns:

  • Integer, nil paginate must equal true page must be !nil?



114
115
116
# File 'lib/queries/query/filter.rb', line 114

def per
  @per
end

#person_queryQuery::Person::Filter?

Returns:

  • (Query::Person::Filter, nil)


177
178
179
# File 'lib/queries/query/filter.rb', line 177

def person_query
  @person_query
end

#project_idArray

Parameters:

  • project_id (Array, Integer)

Returns:

  • (Array)


100
101
102
# File 'lib/queries/query/filter.rb', line 100

def project_id
  @project_id
end

#recentObject

Returns Boolean Applies an order on updated.

Returns:

  • Boolean Applies an order on updated.



181
182
183
# File 'lib/queries/query/filter.rb', line 181

def recent
  @recent
end

#taxon_name_queryQuery::TaxonName::Filter?

Returns:

  • (Query::TaxonName::Filter, nil)


162
163
164
# File 'lib/queries/query/filter.rb', line 162

def taxon_name_query
  @taxon_name_query
end

Class Method Details

.annotator_paramsObject

to be merged into included params

Returns:

  • Array, nil a [:a, :b, []] formatted Array



278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
# File 'lib/queries/query/filter.rb', line 278

def self.annotator_params
  h = nil
  if i = included_annotator_facets
    # Setup with the first
    a = i.shift
    h = a.params

    if !h.last.kind_of?(Hash)
      h << {}
    end

    c = h.last

    # Now do the rest
    i.each do |j|
      p = j.params

      if p.last.kind_of?(Hash)
        c.merge!(p.pop)
      end

      h = p + h
    end
  end
  h
end

.api_except_paramsObject

Any params set here, and in corresponding subclasses will not be permitted when api: true is present



271
272
273
# File 'lib/queries/query/filter.rb', line 271

def self.api_except_params
  []
end

.api_excluded_paramsObject



305
306
307
308
309
# File 'lib/queries/query/filter.rb', line 305

def self.api_excluded_params
  [
    # if there are things like created_by_id that we deem universally out they go here
  ] + api_except_params
end

.included_annotator_facetsObject



237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
# File 'lib/queries/query/filter.rb', line 237

def self.included_annotator_facets
  f = [
    ::Queries::Concerns::Users
  ]

  if referenced_klass.annotates?
    f.push ::Queries::Concerns::Polymorphic if self < ::Queries::Concerns::Polymorphic
  else
    # TODO There is room for an AlternateValue concern here
    f.push ::Queries::Concerns::Attributes if self < ::Queries::Concerns::Attributes
    f.push ::Queries::Concerns::Citations if self < ::Queries::Concerns::Citations
    f.push ::Queries::Concerns::Containable if self < ::Queries::Concerns::Containable
    f.push ::Queries::Concerns::DataAttributes if self < ::Queries::Concerns::DataAttributes
    f.push ::Queries::Concerns::DateRanges if self < ::Queries::Concerns::DateRanges
    f.push ::Queries::Concerns::Depictions if self < ::Queries::Concerns::Depictions
    f.push ::Queries::Concerns::Identifiers if self < ::Queries::Concerns::Identifiers
    f.push ::Queries::Concerns::Notes if self < ::Queries::Concerns::Notes
    f.push ::Queries::Concerns::Protocols if self < ::Queries::Concerns::Protocols
    f.push ::Queries::Concerns::Tags if self < ::Queries::Concerns::Tags
  end

  f
end

.inverted_subqueriesHash

Returns only referenced in specs.

Returns:

  • (Hash)

    only referenced in specs



60
61
62
63
64
65
66
67
68
69
70
71
72
# File 'lib/queries/query/filter.rb', line 60

def self.inverted_subqueries
  r = {}
  SUBQUERIES.each do |k,v|
    v.each do |m|
      if r[m]
        r[m].push k
      else
        r[m] = [k]
      end
    end
  end
  r
end

.paramsObject

Returns Array merges ‘[:a, []]` into [:a].

Returns:

  • Array merges ‘[:a, []]` into [:a]



263
264
265
266
267
# File 'lib/queries/query/filter.rb', line 263

def self.params
  a = self::PARAMS.dup
  b = a.pop.keys
  (a + b).uniq
end

Instance Method Details

#all(nil_empty = false) ⇒ ActiveRecord::Relation

See /lib/queries/ARCHITECTURE.md for additional explanation.

Parameters:

  • nil_empty (Boolean) (defaults to: false)

    If true then if there are no clauses return nil not .all

Returns:

  • (ActiveRecord::Relation)


573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
# File 'lib/queries/query/filter.rb', line 573

def all(nil_empty = false)

  # TODO: should turn off/on project_id here on nil empty?

  a = all_and_clauses
  b = all_merge_clauses

  return nil if nil_empty && a.nil? && b.nil?

  # !! Do not apply `.distinct here`

  q = nil
  if a && b
    q = b.where(a)
  elsif a
    q = referenced_klass.where(a)
  elsif b
    q = b
  else
    q = referenced_klass.all
  end

  if recent
    q = referenced_klass.from(q.all, table.name).order(updated_at: :desc)
  end

  if paginate
    q = q.page(page).per(per)
  end

  q
end

#all_and_clausesActiveRecord::Relation?

Returns:

  • (ActiveRecord::Relation, nil)


532
533
534
535
536
537
538
539
540
541
542
# File 'lib/queries/query/filter.rb', line 532

def all_and_clauses
  clauses = and_clauses + annotator_and_clauses + shared_and_clauses
  clauses.compact!
  return nil if clauses.empty?

  a = clauses.shift
  clauses.each do |b|
    a = a.and(b)
  end
  a
end

#all_merge_clausesScope?

Of interest, the previous native ‘merge()` (and `and()“) make things complicated:

For example a.merge(b) != b.merge(a) in some cases. This is because certain clauses are tossed without warning (notably ‘.from()`). See:

We presently use SQL with INTERSECTION to combine facets.

Returns:

  • (Scope, nil)


560
561
562
563
564
565
566
# File 'lib/queries/query/filter.rb', line 560

def all_merge_clauses
  clauses = merge_clauses + annotator_merge_clauses
  clauses.compact!

  return nil if clauses.empty?
  referenced_klass_intersection(clauses)
end

#and_clausesObject

Defined in inheriting classes



527
528
529
# File 'lib/queries/query/filter.rb', line 527

def and_clauses
  []
end

#annotator_and_clausesObject



504
505
506
507
508
509
510
511
512
513
514
515
516
# File 'lib/queries/query/filter.rb', line 504

def annotator_and_clauses
  a = []
  self.class.included_annotator_facets.each do |c|
    if c.respond_to?(:and_clauses)
      c.and_clauses.each do |f|
        if v = send(f)
          a.push v
        end
      end
    end
  end
  a
end

#annotator_merge_clausesObject



486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
# File 'lib/queries/query/filter.rb', line 486

def annotator_merge_clauses
  a = []

  # !! Interesting `.compact` removes #<ActiveRecord::Relation []>,
  # so patterns that us collect.flatten.compact return empty,
  #  `.present?` fails as well, so verbose loops here
  self.class.included_annotator_facets.each do |c|
    if c.respond_to?(:merge_clauses)
      c.merge_clauses.each do |f|
        if v = send(f)
          a.push v
        end
      end
    end
  end
  a
end

#attribute_exact_facet(attribute = nil) ⇒ Object

params attribute [Symbol]

 a facet for use when params include `author`, and `exact_author` pattern combinations
 See queries/source/filter.rb for example use
 See /spec/lib/queries/source/filter_spec.rb
!! Whitespace (e.g. tabs, newlines) is ignored when exact is used !!!


467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
# File 'lib/queries/query/filter.rb', line 467

def attribute_exact_facet(attribute = nil)
  a = attribute.to_sym
  return nil if send(a).blank?
  if send("exact_#{a}".to_sym)

    # TODO: Think we need to handle ' and '

    v = send(a)
    v.gsub!(/\s+/, ' ')
    v = ::Regexp.escape(v)
    v.gsub!(/\\\s+/, '\s*') # spaces are escaped, but we need to expand them in case we dont' get them caught
    v = '^\s*' + v + '\s*$'

    table[a].matches_regexp(v)
  else
    table[a].matches('%' + send(a).strip.gsub(/\s+/, '%') + '%')
  end
end

#deep_permit(params) ⇒ Object

Returns ActionController::Parameters.

Returns:

  • ActionController::Parameters



407
408
409
# File 'lib/queries/query/filter.rb', line 407

def deep_permit(params)
  p = params.permit( permitted_params(params.to_unsafe_hash))
end

#merge_clausesObject

Defined in inheriting classes



545
546
547
# File 'lib/queries/query/filter.rb', line 545

def merge_clauses
  []
end

#model_id_facetObject

Returns id= facet, automatically added to all queries. Over-ridden in some base classes.



438
439
440
441
442
# File 'lib/queries/query/filter.rb', line 438

def model_id_facet
  m = (base_name + '_id').to_sym
  return nil if send(m).empty?
  table[:id].in(send(m))
end

#object_global_id_facetObject



449
450
451
452
453
454
455
456
457
458
459
460
# File 'lib/queries/query/filter.rb', line 449

def object_global_id_facet
  return nil if object_global_id.empty?
  ids = []
  object_global_id.each do |i|
    g = GlobalID.parse(i)
    # If any global_ids do not reference this Class, abort
    return nil unless g.model_class.base_class.name == referenced_klass.name
    ids.push g.model_id
  end

  table[:id].in(ids)
end

#permitted_params(hsh) ⇒ Object

This method is a preprocessor that discovers, by finding the nested subqueries, which params should be permitted. It is used to build a permitable profile of parameters.

That profile is then used in the actual .permit() call.

An alternate solution, first tried, is to permit the params directly during inspection for subqueries. This also would work, however there are some nice benefits to having a profile of the allowed params available as an Array, for example we can use it for API documentation a little easier(?!).

In essence what we needed was for ActionController::Parameters to be able to accumulate (remember) all permitted params (not just their actual data) over multiple .permit() calls. If we had that, then we could do something like params.permitted_params => Array after multiple calls like params.permit(:a), params.permit(:b).

any parameter set for the query.

Returns:

  • Array like [:a,:b, :c, []]



332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
# File 'lib/queries/query/filter.rb', line 332

def permitted_params(hsh)
  h = self.class::PARAMS.deep_dup
  h.unshift(:per)
  h.unshift(:page)
  h.unshift(:paginate)

  if !h.last.kind_of?(Hash)
    h << {}
  end

  c = h.last # a {}

  if n = self.class.annotator_params
    c.merge!(n.pop)
    h = n + h
  end

  b = subquery_vector(hsh)

  parent = self.class

  while !b.empty?
    a = b.shift

    next unless SUBQUERIES[parent.base_name.to_sym].include?( a.to_s.gsub('_query', '').to_sym )

    q = FILTER_QUERIES[a].safe_constantize
    p = q::PARAMS.deep_dup

    p.unshift(:per)
    p.unshift(:page)
    p.unshift(:paginate)

    if !p.last.kind_of?(Hash)
      p << {}
    end

    if n = q.annotator_params
      p.last.merge!(n.pop)
      p = n + p
    end

    c[a] = p

    c = p.last

    parent = q
  end

  if api
    self.class.api_excluded_params.each do |a|
      h.delete_if{|k,v| a == k}
      h.last.delete_if{|k,v| a == k }
    end
  end

  h
end

#project_id_facetObject



444
445
446
447
# File 'lib/queries/query/filter.rb', line 444

def project_id_facet
  return nil if project_id.empty?
  table[:project_id].in(project_id)
end

#set_nested_queries(params) ⇒ Object

TODO: when a nesting problem is found we need to flag the query as invalid

Returns:

  • True



415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
# File 'lib/queries/query/filter.rb', line 415

def set_nested_queries(params)

  if n = params.select{|k, p| k.to_s =~ /_query/ }
    return nil if n.keys.count != 1 # !!! can't have multiple nested queries inside one level !!! This lets us eliminate infinite loops at the cost of expressiveness?!

    query_name = n.first.first

    return nil unless SUBQUERIES[base_name.to_sym].include?( query_name.to_s.gsub('_query', '').to_sym ) # must be registered

    query_params = n.first.last

    q = FILTER_QUERIES[query_name].safe_constantize.new(query_params)

    # assign to @<model>_query
    v = send("#{query_name}=".to_sym, q)
  end

  true
end

#shared_and_clausesObject



518
519
520
521
522
523
524
# File 'lib/queries/query/filter.rb', line 518

def shared_and_clauses
  [
    project_id_facet,
    model_id_facet,
    object_global_id_facet,
  ]
end

#subquery_vector(hsh) ⇒ Array of Symbol

Since queries nest linearly we don’t need to recursion.

Returns:

  • (Array of Symbol)

    all queries, in nested order



395
396
397
398
399
400
401
402
403
# File 'lib/queries/query/filter.rb', line 395

def subquery_vector(hsh)
  result = []
  while !hsh.keys.select{|k| k =~ /_query/}.empty?
    a = hsh.keys.select{|k| k =~ /_query/}
    result += a
    hsh = hsh[a.first]
  end
  result.map(&:to_sym)
end