Class: SafeDb::EvolveState

Inherits:
Object
  • Object
show all
Defined in:
lib/model/state_evolve.rb

Overview

Cycle cycles state indices and content crypt files to and from master and branches. The need to cycle content occurs during

  • initialization - a new master state box is created

  • login - branch state is created that mirrors master

  • commit - transfers state from branch to master

  • refresh - transfers state from master to branch

Class Method Summary collapse

Class Method Details

.clone_book_into_branch(book_id, branch_id, master_keys, crypt_key) ⇒ Object

When we login to a book which may or may not be the first book in the branch that we have logged into, we are effectively cloning all its master crypts and some of its keys (indices).

To clone a book into a branch we

  • create a branch crypts folder and copy all master crypts into it

  • we create branch indices under general and book_id sections

  • we copy the commit reference and content identifier from the master

  • lock the content crypt key with the branch key and save the ciphertext

commit references

We can only commit (save) a branch’s crypts when the master and branch commit references match. The commit process places a new commit reference into both the master and branch indices. Like git’s push/pull, this prevents a sync when the master has moved forward by one or more commits.

Parameters:

  • book_id (String) —

    the book identifier this branch is about

  • branch_id (String) —

    the identifier pertaining to this branch

  • master_keys (DataMap) —

    keys from the book’s master line

  • crypt_key (Key) —

    symmetric branch content encryption key



318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
 
# File 'lib/model/state_evolve.rb', line 318

def self.clone_book_into_branch( book_id, branch_id, master_keys, crypt_key )

  FileUtils.mkdir_p( FileTree.branch_crypts_folder( book_id, branch_id ) )
  FileUtils.copy_entry( FileTree.master_crypts_folder( book_id ), FileTree.branch_crypts_folder( book_id, branch_id ) )
  branch_keys = create_branch_indices( book_id, branch_id )

  branch_keys.set( Indices::CONTENT_IDENTIFIER, master_keys.get( Indices::CONTENT_IDENTIFIER ) )
  branch_keys.set( Indices::CONTENT_RANDOM_IV,  master_keys.get( Indices::CONTENT_RANDOM_IV  ) )
  branch_keys.set( Indices::COMMIT_IDENTIFIER,   master_keys.get( Indices::COMMIT_IDENTIFIER   ) )

  branch_key = KeyDerivation.regenerate_shell_key( Branch.to_token() )
  key_ciphertext = branch_key.do_encrypt_key( crypt_key )
  branch_keys.set( Indices::CRYPT_CIPHER_TEXT, key_ciphertext )

end

.commit(book) ⇒ Object

In the main, the commit use case changes the master so that it mirrors the branch’s state. A commit syncs the master’s state to mirror the branch.

The Simple Check In

The simplest case is when no other branch has issued a commit since this branch

  • logged in

  • checked in or

  • checked out

In this case the main events are to

  • make the master crypts mirror the branch crypts

  • update the master content ID to mirror the branch

  • give a new commit ID to both master and branch

The Commit ID Lifecycle

A new commit ID is only created during

  • either the first login since the machine booted up

  • or a branch commit

The commit ID is copied from master to branch during

  • either subsequent logins

  • or a branch refresh



174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
 
# File 'lib/model/state_evolve.rb', line 174

def self.commit( book )

# @todo => If mismatch in commit IDs then print message instructing to first do safe refresh

  FileUtils.remove_entry( FileTree.master_crypts_folder( book.book_id() ) )
  FileUtils.mkdir_p( FileTree.master_crypts_folder( book.book_id() ) )
  FileUtils.copy_entry( FileTree.branch_crypts_folder( book.book_id(), book.branch_id() ), FileTree.master_crypts_folder( book.book_id() ) )

  master_keys = DataMap.new( Indices::MASTER_INDICES_FILEPATH )
  master_keys.use( book.book_id() )
  branch_keys = DataMap.new( FileTree.branch_indices_filepath( book.branch_id() ) )
  branch_keys.use( book.book_id() )

  commit_id = Identifier.get_random_identifier( 16 )
  branch_keys.set( Indices::COMMIT_IDENTIFIER, commit_id )
  master_keys.set( Indices::COMMIT_IDENTIFIER, commit_id )

  master_keys.set( Indices::CONTENT_IDENTIFIER, branch_keys.get( Indices::CONTENT_IDENTIFIER ) )
  master_keys.set( Indices::CONTENT_RANDOM_IV,  branch_keys.get( Indices::CONTENT_RANDOM_IV  ) )

  commit_msg = "safe commit for #{book.book_name()} in branch #{book.branch_id()} on #{TimeStamp.readable()}."

  GitFlow.stage( Indices::MASTER_CRYPTS_FOLDER_PATH )
  GitFlow.list( Indices::MASTER_CRYPTS_FOLDER_PATH )
  GitFlow.list( Indices::MASTER_CRYPTS_FOLDER_PATH, true )
  GitFlow.commit( Indices::MASTER_CRYPTS_FOLDER_PATH, commit_msg )

end

.copy_commit_id_to_branch(book) ⇒ Object

Copy the master commit identifier to the branch. This signifies that the branch is aligned (and ready) to commit its changes into the master.

Parameters:

  • book (Book) —

    the book whose commit IDs will be manipulated



234
235
236
237
238
239
240
241
242
243
244
 
# File 'lib/model/state_evolve.rb', line 234

def self.copy_commit_id_to_branch( book )

  master_keys = DataMap.new( Indices::MASTER_INDICES_FILEPATH )
  master_keys.use( book.book_id() )
  branch_keys = DataMap.new( FileTree.branch_indices_filepath( book.branch_id() ) )
  branch_keys.use( book.book_id() )

  master_commit_id = master_keys.get( Indices::COMMIT_IDENTIFIER )
  branch_keys.set( Indices::COMMIT_IDENTIFIER, master_commit_id )

end

.create_book(book_identifier) ⇒ Object

Create the book within the master indices file and set its book identifier along with the initialize time and a fresh commit identifier.

Parameters:

  • book_identifier (String) —

    the identifier of the book to create



270
271
272
273
274
275
276
277
 
# File 'lib/model/state_evolve.rb', line 270

def self.create_book( book_identifier )
  FileUtils.mkdir_p( FileTree.master_crypts_folder( book_identifier ) )

  keypairs = DataMap.new( Indices::MASTER_INDICES_FILEPATH )
  keypairs.use( book_identifier )
  keypairs.set( Indices::SAFE_BOOK_INITIALIZE_TIME, TimeStamp.readable() )
  keypairs.set( Indices::COMMIT_IDENTIFIER, Identifier.get_random_identifier( 16 ) )
end

.create_branch_indices(book_id, branch_id) ⇒ DataMap

Create and return the branch indices DataMap pertaining to both the current book and branch whose ids are given in the first and second parameters.

Parameters:

  • book_id (String) —

    the book identifier this branch is about

  • branch_id (String) —

    the identifier pertaining to this branch

Returns:

  • (DataMap) —

    return the keys pertaining to this branch and book



341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
 
# File 'lib/model/state_evolve.rb', line 341

def self.create_branch_indices( book_id, branch_id )

  branch_exists = File.exists? FileTree.branch_indices_filepath( branch_id )
  branch_keys = DataMap.new( FileTree.branch_indices_filepath( branch_id ) )
  branch_keys.use( Indices::BRANCH_DATA )
  branch_keys.set( Indices::BRANCH_INITIAL_LOGIN_TIME, TimeStamp.readable() ) unless branch_exists
  branch_keys.set( Indices::BRANCH_LAST_ACCESSED_TIME, TimeStamp.readable() )
  branch_keys.set( Indices::CURRENT_BRANCH_BOOK_ID, book_id )

  logged_in = branch_keys.has_section?( book_id )
  branch_keys.use( book_id )
  branch_keys.set( Indices::BOOK_BRANCH_LOGIN_TIME, TimeStamp.readable() ) unless logged_in
  branch_keys.set( Indices::BOOK_LAST_ACCESSED_TIME, TimeStamp.readable() )

  return branch_keys

end

.login(book_keys, human_secret) ⇒ Boolean

We recycle the (kdf) derived key every time we are handed the human password (during init and login) but the high entropy machine generated random key is only recycled at a special time.

When is the high entropy key recycled?

The high entropy key is recycled only on the first login into a book since the machine reboot. This is because subsequent branch logins that protect the random key will need to check back with the master branch when performing either a diff or refresh operations. Also the commit operation must maintain the same content encryption key for readability by validated agents.

Parameters:

  • book_keys (DataMap) —

    the DataMap contains the salts for key rederivation seeing as we have the book password and the rederived key will be able to unlock the ciphertext along with the random initialization vector (iv) also in the key map.

    Unlocking the ciphertext reveals the random high entropy key which can be used for the asymmetric decryption of the content ciphertext which is in a file marked with the content identifier also within the book keys.

  • human_secret (String) —

    the secret text that can potentially be cryptographically weak (low entropy). This text is severely strengthened and morphed into a key using multiple key derivation functions like PBKDF2, BCrypt and SCrypt.

    The secret text is discarded and the derived inter-branch key is used only to encrypt the randomly generated super strong index key, before being itself discarded.

    The key ring only stores the salts. This means the secret text based key can only be regenerated at the next login, which explains the inter-branch label.

Returns:

  • (Boolean) —

    return false if failure decrypting with human password occurs. True is returned if the login logic completes naturally.



60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
 
# File 'lib/model/state_evolve.rb', line 60

def self.( book_keys, human_secret )

  the_book_id = book_keys.section()

  old_human_key = KdfApi.regenerate_from_salts( human_secret, book_keys )
  is_correct_password = old_human_key.can_decrypt_key( book_keys.get( Indices::CRYPT_CIPHER_TEXT ) )
  return false unless is_correct_password

  the_crypt_key = old_human_key.do_decrypt_key( book_keys.get( Indices::CRYPT_CIPHER_TEXT ) )
  plain_content = Content.unlock_master( the_crypt_key, book_keys )

  remove_crypt_path = FileTree.master_crypts_filepath( the_book_id, book_keys.get( Indices::CONTENT_IDENTIFIER ) )

   = StateInspect.is_first_login?( book_keys )
  the_crypt_key = Key.from_random if 
  recycle_keys( the_crypt_key, the_book_id, human_secret, book_keys, plain_content )
  set_bootup_id( book_keys ) if 

  create_crypt_path = FileTree.master_crypts_filepath( the_book_id, book_keys.get( Indices::CONTENT_IDENTIFIER ) )
  branch_id = Identifier.derive_branch_id( Branch.to_token() )
  commit_msg = "safe login to #{the_book_id} at branch #{branch_id} on #{TimeStamp.readable()}."

  # Remove the master chapter crypt file from the local git repository and add
  # the new master chapter crypt to the local git repository.
  GitFlow.del_file( Indices::MASTER_CRYPTS_FOLDER_PATH, remove_crypt_path )
  GitFlow.add_file( Indices::MASTER_CRYPTS_FOLDER_PATH, create_crypt_path )
  GitFlow.add_file( Indices::MASTER_CRYPTS_FOLDER_PATH, Indices::MASTER_INDICES_FILEPATH )
  GitFlow.list( Indices::MASTER_CRYPTS_FOLDER_PATH )
  GitFlow.list( Indices::MASTER_CRYPTS_FOLDER_PATH, true )
  GitFlow.commit( Indices::MASTER_CRYPTS_FOLDER_PATH, commit_msg )

  clone_book_into_branch( the_book_id, branch_id, book_keys, the_crypt_key )

  return true

end

.recycle_both_keys(book_id, human_secret, data_map, content_body) ⇒ Key

This method creates a new high entropy content encryption key and then forwards it on to behaviour that recycles the (kdf) key from the provided human sourced secret.

Parameters:

  • book_id (String) —

    the identifier of the book whose keys we are cycling

  • human_secret (String) —

    this secret is sourced into key derivation functions

  • data_map (Hash) —

    book related key/value data that will be populated as appropriate

  • content_body (String) —

    this content is encrypted and the ciphertext output stored

Returns:

  • (Key) —

    the generated random high entropy key that the content is locked with



109
110
111
 
# File 'lib/model/state_evolve.rb', line 109

def self.recycle_both_keys( book_id, human_secret, data_map, content_body )
  recycle_keys( Key.from_random(), book_id, human_secret, data_map, content_body )
end

.recycle_keys(high_entropy_key, book_id, human_secret, data_map, content_body) ⇒ Object

During initialization or login we recycle keys produced by key derivation functions (BCrypt. SCrypt and/or PBKDF2) from human sourced secrets.

The flow of events of the recycling process is to

  • use the random high entropy key given in parameter one

  • lock the provided content with this high entropy key

  • save ciphertext in a file named by a random identifier

  • write this random identifier to the key cache

  • write the initialization vector to the key cache

  • use KDFs to derive a key from the human sourced password

  • save the salts crucial for reproducing this derived key

  • use the derived key to encrypt the high entropy key

  • write the resulting ciphertext into the key cache

  • return the high entropy key that locked the content

Parameters:

  • high_entropy_key (Key) —

    the machine generated high entropy content encryption key

  • book_id (String) —

    the identifier of the book whose keys we are cycling

  • human_secret (String) —

    this secret is sourced into key derivation functions

  • data_map (Hash) —

    book related key/value data that will be populated as appropriate

  • content_body (String) —

    this content is encrypted and the ciphertext output stored



136
137
138
139
140
141
142
 
# File 'lib/model/state_evolve.rb', line 136

def self.recycle_keys( high_entropy_key, book_id, human_secret, data_map, content_body )

  Content.lock_master( book_id, high_entropy_key, data_map, content_body )
  derived_key = KdfApi.generate_from_password( human_secret, data_map )
  data_map.set( Indices::CRYPT_CIPHER_TEXT, derived_key.do_encrypt_key( high_entropy_key ) )

end

.refresh(book) ⇒ Object

A refresh merges down the master’s data into the data of this working branch. The commit ID of the working branch after the refresh is made to be equivalent with that of the master. This act signifies that a commit is now allowed (as long as another branch doesn’t commit in the meantime).

Parameters:

  • book (Book) —

    the book whose master data will be merged down into the branch.



211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
 
# File 'lib/model/state_evolve.rb', line 211

def self.refresh( book )

  master_data = book.to_master_data()
  branch_data = book.to_branch_data()

  merged_verse_count = 0
  master_data.each_pair do | chapter_name, chapter_data |
    book.import_chapter( chapter_name, chapter_data )
    merged_verse_count += chapter_data.length()
  end

  book.write()

  puts ""
  puts "#{master_data.length()} chapters and #{merged_verse_count} verses from master were merged in.\n"
  puts ""

end

.set_bootup_id(data_map) ⇒ Object

Set the booup identifier within the parameter key/value map under the globally recognized Indices::BOOTUP_IDENTIFIER constant. This method expects the DataMap section name to be a significant identifier.

Parameters:

  • data_map (DataMap) —

    the data map in which we set the bootup id



252
253
254
255
256
257
258
259
260
261
262
263
 
# File 'lib/model/state_evolve.rb', line 252

def self.set_bootup_id( data_map )

  has_bootup_id = data_map.contains?( Indices::BOOTUP_IDENTIFIER )
  old_bootup_id = data_map.get( Indices::BOOTUP_IDENTIFIER ) if has_bootup_id
  log.info(x) { "overriding bootup id [#{old_bootup_id}] in section [#{data_map.section()}]." } if has_bootup_id

  new_bootup_id = MachineId.get_bootup_id()
  data_map.set( Indices::BOOTUP_IDENTIFIER, new_bootup_id )
  log.info(x) { "setting bootup id in section [#{data_map.section()}] to [#{new_bootup_id}]." }
  MachineId.log_reboot_times()

end

.use_book(book_id) ⇒ Object

Switch the current branch (if necessary) to using the book whose ID is specified in the parameter. Only call method if we are definitely in a logged in state.

Parameters:

  • book_id (String) —

    book identifier that login request is against



285
286
287
288
289
290
291
292
 
# File 'lib/model/state_evolve.rb', line 285

def self.use_book( book_id )
  branch_id = Identifier.derive_branch_id( Branch.to_token() )
  branch_keys = DataMap.new( FileTree.branch_indices_filepath( branch_id ) )
  branch_keys.use( Indices::BRANCH_DATA )
  current_book_id = branch_keys.get( Indices::CURRENT_BRANCH_BOOK_ID )
  log.info(x) { "Current book is #{current_book_id} and the instruction is to use #{book_id}" }
  branch_keys.set( Indices::CURRENT_BRANCH_BOOK_ID, book_id )
end