Class: Cicada::CicadaMain

Inherits:
Object
  • Object
show all
Includes:
IATScripting
Defined in:
lib/cicada/cicada_main.rb

Overview

This class is the main entry point for running 3D high-resolution colocalization and CICADA.

Constant Summary collapse

REQUIRED_PARAMETERS =

parameters required by the methods in this class

[:dirname, :basename, :im_border_size, :half_z_size, :determine_correction, :pixelsize_nm, :z_sectionsize_nm, :num_wavelengths, :photons_per_greylevel]
OPTIONAL_PARAMETERS =

parmeters used but not required in this class or only required for optional functionality

[:precomputed_position_data, :max_threads, :darkcurrent_image, :residual_cutoff, :max_greylevel_cutoff, :distance_cutoff, :fit_error_cutoff, :determine_correction, :determine_tre, :output_positions_to_directory, :in_situ_aberr_corr_basename, :in_situ_aberr_corr_channel, :log_to_file, :log_detailed_messages]
FAILURE_REASONS = 
{r2: "R^2 value", edge: "Too close to image edge", sat: "Saturated pixels", sep: "Separation between channels too large", err: "Fit error too large"}

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(p) ⇒ CicadaMain

Sets up the analysis from a parameter dictionary.

Parameters:

  • p (ParameterDictionary, Hash) —

    a parameter dictionary or other object with hash-like behavior containing all the parameters for the analysis.



77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
 
# File 'lib/cicada/cicada_main.rb', line 77

def initialize(p)
  
  @parameters = p

  @parameters = RImageAnalysisTools.create_parameter_dictionary(p) unless @parameters.is_a? ParameterDictionary

  @failures = {r2: 0, edge: 0, sat: 0, sep: 0, err: 0}

  if @parameters[:darkcurrent_image] then

    @dark_image = FileInteraction.load_image(@parameters[:darkcurrent_image])

  end

  set_up_logging
  
end

Instance Attribute Details

#failures ⇒ Object

Returns the value of attribute failures.



67
68
69
 
# File 'lib/cicada/cicada_main.rb', line 67

def failures
  @failures
end

#logger ⇒ Object

Returns the value of attribute logger.



67
68
69
 
# File 'lib/cicada/cicada_main.rb', line 67

def logger
  @logger
end

#parameters ⇒ Object

Returns the value of attribute parameters.



67
68
69
 
# File 'lib/cicada/cicada_main.rb', line 67

def parameters
  @parameters
end

Class Method Details

.parse_parameter_file(fn) ⇒ ParameterDictionary

Reads a parameters file and creates a parameter dictionary.

Parameters:

  • fn (String) —

    the filename of the parameter file

Returns:

  • (ParameterDictionary) —

    the parsed parameters



670
671
672
673
674
675
676
677
678
 
# File 'lib/cicada/cicada_main.rb', line 670

def self.parse_parameter_file(fn)

  java_import Java::edu.stanford.cfuller.imageanalysistools.meta.AnalysisMetadataParserFactory

  parser = AnalysisMetadataParserFactory.createParserForFile(fn)

  parser.parseFileToParameterDictionary(fn)

end

.run_from_parameter_file(fn) ⇒ void

This method returns an undefined value.

Runs analysis using a specified parameter file.

Parameters:

  • fn (String) —

    the filename of the parameter file



687
688
689
690
691
692
693
694
695
 
# File 'lib/cicada/cicada_main.rb', line 687

def self.run_from_parameter_file(fn)

  p = parse_parameter_file(fn)

  c = new(p)

  c.go

end

Instance Method Details

#check_edges(to_check) ⇒ Boolean

Checks whether the fitted position is too close to the image edges.

Parameters:

  • to_check (ImageObject) —

    the ImageObject to check for fitting success

Returns:

  • (Boolean) —

    whether the fitting was successful by this criterion.



287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
 
# File 'lib/cicada/cicada_main.rb', line 287

def check_edges(to_check)

  eps = 0.1

  border_size = @parameters[:im_border_size].to_f
  z_size = @parameters[:half_z_size].to_f

  range_x = border_size...(to_check.getParent.getDimensionSizes[:x] - border_size)
  range_y = border_size...(to_check.getParent.getDimensionSizes[:y] - border_size)
  range_z = z_size...(to_check.getParent.getDimensionSizes[:z] - z_size)

  to_check.getFitParametersByChannel.each do |fp|
    
    x = fp.getPosition(ImageCoordinate::X)
    y = fp.getPosition(ImageCoordinate::Y)
    z = fp.getPosition(ImageCoordinate::Z)

    ok = (range_x.include?(x) and range_y.include?(y) and (range_z.include?(z) or to_check.getParent.getDimensionSizes[:z] == 1))

    unless ok then
      
      @failures[:edge] += 1

      @logger.debug { "check failed for object #{to_check.getLabel} position: #{x}, #{y}, #{z}" }

      return false

    end

  end

  true

end

#check_error(to_check) ⇒ Boolean

Checks whether the caluclated fitting error (summed in quadrature over all wavelengths) is larger than a specified cutoff.

Parameters:

  • to_check (ImageObject) —

    the ImageObject to check for fitting success

Returns:

  • (Boolean) —

    whether the fitting was successful by this criterion.



415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
 
# File 'lib/cicada/cicada_main.rb', line 415

def check_error(to_check)

  if @parameters[:fit_error_cutoff] then

    total_error = 0

    to_check.getFitErrorByChannel.each do |d|

      total_error += d**2

    end

    total_error = total_error**0.5

    if total_error > @parameters[:fit_error_cutoff].to_f or total_error.nan? then

      @failures[:err] += 1

      @logger.debug { "check failed for object #{to_check.getLabel} with total fitting error: #{total_error}" }

      return false

    end

  end

  true

end

#check_fit(to_check) ⇒ Boolean

Checks whether the fitting was successful for a given object according to several criteria: whether the fitting finished without error, whether the R^2 value of the fit is above the cutoff, whether the object is too close to the image edges, whether the camera is saturated in the object, whether the separation between channels is above some cutoff, and whether the calculated fitting error is too large. Cutoffs for all these criteria are specified in the parameters file.

Parameters:

  • to_check (ImageObject) —

    the ImageObject to check for fitting success

Returns:

  • (Boolean) —

    whether the fitting was successful by all criteria.



245
246
247
248
249
250
251
 
# File 'lib/cicada/cicada_main.rb', line 245

def check_fit(to_check)

  checks = [:check_r2, :check_edges, :check_saturation, :check_separation, :check_error]

  to_check.finishedFitting and checks.all? { |c| self.send(c, to_check) }

end

#check_r2(to_check) ⇒ Boolean

Checks whether the fit R^2 value is below the specified cutoff.

Parameters:

  • to_check (ImageObject) —

    the ImageObject to check for fitting success

Returns:

  • (Boolean) —

    whether the fitting was successful by this criterion.



260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
 
# File 'lib/cicada/cicada_main.rb', line 260

def check_r2(to_check)

  return true unless @parameters[:residual_cutoff]

  to_check.getFitR2ByChannel.each do |r2|

    if r2 < @parameters[:residual_cutoff].to_f then
      
      @failures[:r2] += 1

      @logger.debug { "check failed for object #{to_check.getLabel} R^2 = #{r2}" }

      return false

    end

  end

  true

end

#check_saturation(to_check) ⇒ Boolean

Checks whether the camera has saturated in the object.

Parameters:

  • to_check (ImageObject) —

    the ImageObject to check for fitting success

Returns:

  • (Boolean) —

    whether the fitting was successful by this criterion.



328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
 
# File 'lib/cicada/cicada_main.rb', line 328

def check_saturation(to_check)

  if @parameters[:max_greylevel_cutoff] then

    to_check.boxImages

    cutoff = @parameters[:max_greylevel_cutoff].to_f

    to_check.getParent.each do |ic|

      if to_check.getParent[ic] > cutoff then 

        to_check.unboxImages
        @failures[:sat] += 1

        @logger.debug { "check failed for object #{to_check.getLabel} greylevel: #{to_check.getParent[ic]}" }

        return false 

      end

    end

  end
    
  true

end

#check_separation(to_check) ⇒ Boolean

Checks whether the separation between channels is too large.

Note that this check can significantly skew the distance measurements if the cutoff is too small. This remains here because occasionally closely spaced objects are fit as a single object and produce ridiculous values.

Parameters:

  • to_check (ImageObject) —

    the ImageObject to check for fitting success

Returns:

  • (Boolean) —

    whether the fitting was successful by this criterion.



367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
 
# File 'lib/cicada/cicada_main.rb', line 367

def check_separation(to_check)

  if @parameters[:distance_cutoff] then

    size_c = to_check.getFitParametersByChannel.size

    xy_pixelsize_2 = @parameters[:pixelsize_nm].to_f**2

    z_sectionsize_2 = @parameters[:z_sectionsize_nm].to_f**2

    0.upto(size_c-1) do |ci|
      0.upto(size_c-1) do |cj|

        fp1 = to_check.getFitParametersByChannel.get(ci)
        fp2 = to_check.getFitParametersByChannel.get(cj)

        ijdist = xy_pixelsize_2 * (fp1.getPosition(ImageCoordinate::X) - fp2.getPosition(ImageCoordinate::X))**2 +
          xy_pixelsize_2 * (fp1.getPosition(ImageCoordinate::Y) - fp2.getPosition(ImageCoordinate::Y))**2 +
          z_sectionsize_2 * (fp1.getPosition(ImageCoordinate::Z) - fp2.getPosition(ImageCoordinate::Z))**2

        ijdist = ijdist**0.5

        if (ijdist > @parameters[:distance_cutoff].to_f) then
          
          @failures[:sep] += 1
          @logger.debug { "check failed for object #{to_check.getLabel} with distance: #{ijdist}" } 

          return false              

        end

      end
    end

  end

  true

end

#do_and_save_fits ⇒ List<ImageObject>

Fits all objects in an image or loads objects from disk if they have already been fit and refitting has not been requested.

Saves fits to disk in the parameter-specified data directory.

Returns:

  • (List<ImageObject>) —

    the fitted or loaded image objects



547
548
549
550
551
552
553
554
555
 
# File 'lib/cicada/cicada_main.rb', line 547

def do_and_save_fits

  image_objects = load_or_fit_image_objects

  FileInteraction.write_position_data(image_objects, @parameters)

  image_objects

end

#fit_objects_in_single_image(im_set) ⇒ Array<ImageObject>

Fits all the image objects in a single supplied image.

Does not check whether the fitting was successful.

Parameters:

  • im_set (OpenStruct, #image, #mask) —

    An object that references the image and the mask from which the objects will be fit. Should respond to #image and #mask.

Returns:

  • (Array<ImageObject>) —

    an array containing all the image objects in the image (one per unique greylevel in the mask).



183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
 
# File 'lib/cicada/cicada_main.rb', line 183

def fit_objects_in_single_image(im_set)

  objs = []

  load_and_dark_correct_image(im_set)

  unless im_set.image and im_set.mask then
    
    logger.error { "Unable to process image #{im_set.image_fn}." }

    return objs

  end

  h = Histogram.new(im_set.mask)

  max_threads = 1

  if @parameters[:max_threads] then
    max_threads = @parameters[:max_threads].to_i
  end

  thread_queue = Executors.newFixedThreadPool(max_threads)

  1.upto(h.getMaxValue) do |i|

    obj = GaussianImageObject.new(i, image_shallow_copy(im_set.mask), image_shallow_copy(im_set.image), ParameterDictionary.new(@parameters))

    obj.setImageID(im_set.image_fn)

    objs << obj

  end

  objs.each do |obj|

    submit_single_object(obj, thread_queue)

  end

  thread_queue.shutdown

  until thread_queue.isTerminated do
    sleep 0.4
  end

  objs

end

#get_scalar_diffs_from_vector(vector_diffs) ⇒ Array

Converts an array of vectors to an array of scalars by taking their 2-norm.

Parameters:

  • vector_diffs (Enumerable< Enumerable<Numeric> >) —

    an array of arrays (vectors, etc.) each of which will be normed.

Returns:

  • (Array) —

    an array of the norms of the vectors provided.



652
653
654
655
656
657
658
659
660
 
# File 'lib/cicada/cicada_main.rb', line 652

def get_scalar_diffs_from_vector(vector_diffs)

  vector_diffs.map do |vd|

    Math.sqrt(Math.sum(vd) { |e| e**2 })

  end

end

#go ⇒ void

This method returns an undefined value.

Runs the analysis.



563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
 
# File 'lib/cicada/cicada_main.rb', line 563

def go

  image_objects = do_and_save_fits
  
  pc = PositionCorrector.new(@parameters)
  pc.logger= @logger

  c = pc.generate_correction(image_objects)

  tre = 0.0

  if @parameters[:determine_tre] and @parameters[:determine_correction] then
    
    self.logger.info("calculating tre")
    
    tre = pc.determine_tre(image_objects)

    c.tre= tre

  else

    tre = c.tre

  end

  
  c.write_to_file(FileInteraction.correction_filename(@parameters))

  

  diffs = pc.apply_correction(c, image_objects)

  corrected_image_objects = []

  image_objects.each do |iobj|

    if iobj.getCorrectionSuccessful then
      
      corrected_image_objects << iobj

    end

  end

  FileInteraction.write_position_data(corrected_image_objects, @parameters)

  
  image_objects = corrected_image_objects

  df= P3DFitter.new(@parameters)

  fitparams = df.fit(image_objects, diffs)

  @logger.info { "p3d fit parameters: #{fitparams.join(', ')}" }

  if @parameters[:in_situ_aberr_corr_basename] and @parameters[:in_situ_aberr_corr_channel] then

    slopes = pc.determine_in_situ_aberration_correction

    vector_diffs = pc.apply_in_situ_aberration_correction(image_objects, slopes)

    scalar_diffs = get_scalar_diffs_from_vector(vector_diffs)

    corr_fit_params = df.fit(image_objects, scalar_diffs)

    FileInteraction.write_differences(diffs, @parameters)

    if corr_fit_params then

      @logger.info { "p3d fit parameters after in situ correction: #{fitparams.join(', ') }" }
            
    else

      @logger.info { "unable to fit after in situ correction" } 

    end

  end

end

#load_and_dark_correct_image(im_set) ⇒ void

This method returns an undefined value.

Loads the image and mask from an image and mask pair and darkcurrent corrects the image if specified in the parameters.

Parameters:

  • im_set (OpenStruct, #image_fn, #mask_fn, #image=, #mask=) —

    An object that specified the filename of image and mask and can store the loaded image and mask. Should respond to #image_fn, #mask_fn, #image=, and #mask= for getting the filenames and setting the loaded images, respectively.



126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
 
# File 'lib/cicada/cicada_main.rb', line 126

def load_and_dark_correct_image(im_set)

  im_set.image = FileInteraction.load_image(im_set.image_fn)
  im_set.mask = FileInteraction.load_image(im_set.mask_fn)
  
  if (@dark_image) then

    im_set.image = im_set.image.writableInstance

    isf = ImageSubtractionFilter.new

    isf.setSubtractPlanarImage(true)

    isf.setReferenceImage(@dark_image)
    isf.apply(im_set.image)

  end
  
end

#load_or_fit_image_objects ⇒ Array<ImageObject>

Loads previously existing image objects for the current images or fits them anew if they don’t exist or this is requested in the parameters.

Returns:

  • (Array<ImageObject>) —

    The image objects that have been loaded or fit. Only successfully fit objects that have passed all checks are included.



482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
 
# File 'lib/cicada/cicada_main.rb', line 482

def load_or_fit_image_objects

  image_objects = load_position_data

  unless image_objects then

    image_objects = []

    to_process = FileInteraction.list_files(@parameters)

    to_process.each do |im_set|
      
      objs = fit_objects_in_single_image(im_set)

      objs.each do |o|
        
        if check_fit(o) then

          image_objects << o

        end

        o.nullifyImages

      end

    end

    log_fitting_failures
    
  end

  puts "number of image objects: #{image_objects.size}"

  image_objects

end

#load_position_data ⇒ Array<ImageObject>

Load the position data from disk if this is requested in the specified parameters. If this has not been requested or if the position data file does not exist, returns nil.

Returns:

  • (Array<ImageObject>) —

    the image objects, complete with their fitted positions, or nil if this should be recalculated or if the file cannot be found.



103
104
105
106
107
108
109
110
111
112
113
 
# File 'lib/cicada/cicada_main.rb', line 103

def load_position_data

  if @parameters[:precomputed_position_data] and FileInteraction.position_file_exists?(@parameters) then
    
    return FileInteraction.read_position_data(@parameters)

  end

  nil

end

#log_fitting_failures ⇒ void

This method returns an undefined value.

Formats the fitting failures using their description strings and logs them



526
527
528
529
530
531
532
533
534
535
536
 
# File 'lib/cicada/cicada_main.rb', line 526

def log_fitting_failures

  @logger.info { "fitting failures by type:" }

  @failures.each_key do |k|

    @logger.info { FAILURE_REASONS[k] + ": " + @failures[k].to_s }

  end

end

#set_up_logging ⇒ void

This method returns an undefined value.

Sets up a logger to either standard output or a file with appropriate detail level as specified in the parameters



451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
 
# File 'lib/cicada/cicada_main.rb', line 451

def set_up_logging

  if @parameters[:log_to_file] then

    @logger = Logger.new(@parameters[:log_to_file])

  else

    @logger = Logger.new(STDOUT)

  end

  if @parameters[:log_detailed_messages] then
    
    @logger.sev_threshold = Logger::DEBUG

  else

    @logger.sev_threshold = Logger::INFO

  end

end

#submit_single_object(obj, queue) ⇒ void

This method returns an undefined value.

Submits a single object to a thread queue for fitting.

Parameters:

  • obj (ImageObject) —

    the image object to fit

  • queue (ExecutorService) —

    the thread queue



154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
 
# File 'lib/cicada/cicada_main.rb', line 154

def submit_single_object(obj, queue)

  queue.submit do 
    
      @logger.debug { "Processing object #{obj.getLabel}" }

      begin

        obj.fitPosition(@parameters)

      rescue => e

        logger.error { "error while processing object #{obj.label}: #{e.message}" }

      end

  end

end