Class: Google::Cloud::Datastore::Dataset

Inherits:
Object
  • Object
show all
Defined in:
lib/google/cloud/datastore/dataset.rb,
lib/google/cloud/datastore/dataset/query_results.rb,
lib/google/cloud/datastore/dataset/lookup_results.rb

Overview

# Dataset

Dataset is the data saved in a project’s Datastore. Dataset is analogous to a database in relational database world.

Google::Cloud::Datastore::Dataset is the main object for interacting with Google Datastore. Entity objects are created, read, updated, and deleted by Google::Cloud::Datastore::Dataset.

See Google::Cloud#datastore

Examples:

require "google/cloud/datastore"

datastore = Google::Cloud::Datastore.new

query = datastore.query("Task").
  where("done", "=", false)

tasks = datastore.run query

Direct Known Subclasses

Transaction

Defined Under Namespace

Classes: LookupResults, QueryResults

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(service) ⇒ Dataset

See Google::Cloud#datastore



64
65
66
 
# File 'lib/google/cloud/datastore/dataset.rb', line 64

def initialize service
  @service = service
end

Instance Attribute Details

#service ⇒ Object 



58
59
60
 
# File 'lib/google/cloud/datastore/dataset.rb', line 58

def service
  @service
end

Class Method Details

.default_project ⇒ Object 



87
88
89
90
91
92
93
 
# File 'lib/google/cloud/datastore/dataset.rb', line 87

def self.default_project
  ENV["DATASTORE_DATASET"] ||
    ENV["DATASTORE_PROJECT"] ||
    ENV["GOOGLE_CLOUD_PROJECT"] ||
    ENV["GCLOUD_PROJECT"] ||
    Google::Cloud::Core::Environment.project_id
end

Instance Method Details

#allocate_ids(incomplete_key, count = 1) ⇒ Array<Google::Cloud::Datastore::Key>

Generate IDs for a Key before creating an entity.

Examples:

task_key = datastore.key "Task"
task_keys = datastore.allocate_ids task_key, 5

Parameters:

  • incomplete_key (Key) —

    A Key without ‘id` or `name` set.

  • count (String) (defaults to: 1) —

    The number of new key IDs to create.

Returns:



107
108
109
110
111
112
113
114
115
116
117
 
# File 'lib/google/cloud/datastore/dataset.rb', line 107

def allocate_ids incomplete_key, count = 1
  if incomplete_key.complete?
    fail Google::Cloud::Datastore::KeyError,
         "An incomplete key must be provided."
  end

  ensure_service!
  incomplete_keys = count.times.map { incomplete_key.to_grpc }
  allocate_res = service.allocate_ids(*incomplete_keys)
  allocate_res.keys.map { |key| Key.from_grpc key }
end

#commit {|commit| ... } ⇒ Array<Google::Cloud::Datastore::Entity>

Make multiple changes in a single commit.

Examples:

datastore = Google::Cloud::Datastore.new
datastore.commit do |c|
  c.save task3, task4
  c.delete task1, task2
end

Yields:

  • (commit) —

    a block for making changes

Yield Parameters:

  • commit (Commit) —

    The object that changes are made on

Returns:



260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
 
# File 'lib/google/cloud/datastore/dataset.rb', line 260

def commit
  return unless block_given?
  c = Commit.new
  yield c

  ensure_service!
  commit_res = service.commit c.mutations
  entities = c.entities
  returned_keys = commit_res.mutation_results.map(&:key)
  returned_keys.each_with_index do |key, index|
    next if entities[index].nil?
    entities[index].key = Key.from_grpc(key) unless key.nil?
  end
  entities.each { |e| e.key.freeze unless e.persisted? }
  entities
end

#delete(*entities_or_keys) ⇒ Boolean

Remove entities from the Datastore.

Examples:

datastore = Google::Cloud::Datastore.new
datastore.delete task1, task2

Parameters:

  • entities_or_keys (Entity, Key) —

    One or more Entity or Key objects to remove.

Returns:

  • (Boolean) —

    Returns ‘true` if successful



239
240
241
242
 
# File 'lib/google/cloud/datastore/dataset.rb', line 239

def delete *entities_or_keys
  commit { |c| c.delete(*entities_or_keys) }
  true
end

#entity(*key_or_path, project: nil, namespace: nil) {|entity| ... } ⇒ Google::Cloud::Datastore::Entity

Create a new empty Entity instance. This is a convenience method to make the creation of Entity objects easier.

Examples:

task = datastore.entity

The previous example is equivalent to:

task = Google::Cloud::Datastore::Entity.new

The key can also be passed in as an object:

task_key = datastore.key "Task", "sampleTask"
task = datastore.entity task_key

Or the key values can be passed in as parameters:

task = datastore.entity "Task", "sampleTask"

The previous example is equivalent to:

task_key = Google::Cloud::Datastore::Key.new "Task", "sampleTask"
task = Google::Cloud::Datastore::Entity.new
task.key = task_key

The newly created entity can also be configured using block:

task = datastore.entity "Task", "sampleTask" do |t|
  t["type"] = "Personal"
  t["done"] = false
  t["priority"] = 4
  t["description"] = "Learn Cloud Datastore"
end

The previous example is equivalent to:

task_key = Google::Cloud::Datastore::Key.new "Task", "sampleTask"
task = Google::Cloud::Datastore::Entity.new
task.key = task_key
task["type"] = "Personal"
task["done"] = false
task["priority"] = 4
task["description"] = "Learn Cloud Datastore"

Parameters:

  • key_or_path (Key, Array<Array(String,(String|Integer|nil))>) —

    An optional list of pairs for the key’s path. Each pair may include the key’s kind (String) and an id (Integer) or name (String). This is optional.

  • project (String) (defaults to: nil) —

    The project of the Key. This is optional.

  • namespace (String) (defaults to: nil) —

    namespace kind of the Key. This is optional.

Yields:

  • (entity) —

    a block yielding a new entity

Yield Parameters:

  • entity (Entity) —

    the newly created entity object

Returns:



629
630
631
632
633
634
635
636
637
638
639
640
641
642
 
# File 'lib/google/cloud/datastore/dataset.rb', line 629

def entity *key_or_path, project: nil, namespace: nil
  entity = Entity.new

  # Set the key
  if key_or_path.flatten.first.is_a? Google::Cloud::Datastore::Key
    entity.key = key_or_path.flatten.first
  else
    entity.key = key key_or_path, project: project, namespace: namespace
  end

  yield entity if block_given?

  entity
end

#find(key_or_kind, id_or_name = nil, consistency: nil) ⇒ Google::Cloud::Datastore::Entity? Also known as: get

Retrieve an entity by key.

Examples:

Finding an entity with a key:

task_key = datastore.key "Task", "sampleTask"
task = datastore.find task_key

Finding an entity with a ‘kind` and `id`/`name`:

task = datastore.find "Task", "sampleTask"

Parameters:

  • key_or_kind (Key, String) —

    A Key object or ‘kind` string value.

  • id_or_name (Integer, String, nil) (defaults to: nil) —

    The Key’s ‘id` or `name` value if a `kind` was provided in the first parameter.

  • consistency (Symbol) (defaults to: nil) —

    The non-transactional read consistency to use. Cannot be set to ‘:strong` for global queries. Accepted values are `:eventual` and `:strong`.

    The default consistency depends on the type of lookup used. See [Eventual Consistency in Google Cloud Datastore](cloud.google.com/datastore/docs/articles/balancing-strong-and-eventual-consistency-with-google-cloud-datastore/#h.tf76fya5nqk8) for more information.

Returns:



301
302
303
304
305
306
307
 
# File 'lib/google/cloud/datastore/dataset.rb', line 301

def find key_or_kind, id_or_name = nil, consistency: nil
  key = key_or_kind
  unless key.is_a? Google::Cloud::Datastore::Key
    key = Key.new key_or_kind, id_or_name
  end
  find_all(key, consistency: consistency).first
end

#find_all(*keys, consistency: nil) ⇒ Google::Cloud::Datastore::Dataset::LookupResults Also known as: lookup

Retrieve the entities for the provided keys. The order of results is undefined and has no relation to the order of ‘keys` arguments.

Examples:

datastore = Google::Cloud::Datastore.new

task_key1 = datastore.key "Task", "sampleTask1"
task_key2 = datastore.key "Task", "sampleTask2"
tasks = datastore.find_all task_key1, task_key2

Parameters:

Returns:



333
334
335
336
337
338
339
 
# File 'lib/google/cloud/datastore/dataset.rb', line 333

def find_all *keys, consistency: nil
  ensure_service!
  check_consistency! consistency
  lookup_res = service.lookup(*Array(keys).flatten.map(&:to_grpc),
                              consistency: consistency)
  LookupResults.from_grpc lookup_res, service, consistency
end

#gql(query, bindings = {}) ⇒ Google::Cloud::Datastore::GqlQuery

Create a new GqlQuery instance. This is a convenience method to make the creation of GqlQuery objects easier.

Examples:

gql_query = datastore.gql "SELECT * FROM Task WHERE done = @done",
                          done: false
tasks = datastore.run gql_query

The previous example is equivalent to:

gql_query = Google::Cloud::Datastore::GqlQuery.new
gql_query.query_string = "SELECT * FROM Task WHERE done = @done"
gql_query.named_bindings = {done: false}
tasks = datastore.run gql_query

Parameters:

  • query (String) —

    The GQL query string.

  • bindings (Hash) (defaults to: {}) —

    Named bindings for the GQL query string, each key must match regex ‘[A-Za-z_$]*`, must not match regex `__.*__`, and must not be `“”`. The value must be an `Object` that can be stored as an Entity property value, or a `Cursor`.

Returns:



510
511
512
513
514
515
 
# File 'lib/google/cloud/datastore/dataset.rb', line 510

def gql query, bindings = {}
  gql = GqlQuery.new
  gql.query_string = query
  gql.named_bindings = bindings unless bindings.empty?
  gql
end

#insert(*entities) ⇒ Array<Google::Cloud::Datastore::Entity>

Insert one or more entities to the Datastore. An InvalidArgumentError will raised if the entities cannot be inserted.

Examples:

Insert a new entity:

task = datastore.entity "Task" do |t|
  t["type"] = "Personal"
  t["done"] = false
  t["priority"] = 4
  t["description"] = "Learn Cloud Datastore"
end
task.key.id #=> nil
datastore.insert task
task.key.id #=> 123456

Insert multiple new entities in a batch:

task1 = datastore.entity "Task" do |t|
  t["type"] = "Personal"
  t["done"] = false
  t["priority"] = 4
  t["description"] = "Learn Cloud Datastore"
end

task2 = datastore.entity "Task" do |t|
  t["type"] = "Personal"
  t["done"] = false
  t["priority"] = 5
  t["description"] = "Integrate Cloud Datastore"
end

task_key1, task_key2 = datastore.insert(task1, task2).map(&:key)

Parameters:

  • entities (Entity) —

    One or more entity objects to be inserted.

Returns:



200
201
202
 
# File 'lib/google/cloud/datastore/dataset.rb', line 200

def insert *entities
  commit { |c| c.insert(*entities) }
end

#key(*path, project: nil, namespace: nil) ⇒ Google::Cloud::Datastore::Key

Create a new Key instance. This is a convenience method to make the creation of Key objects easier.

Examples:

task_key = datastore.key "Task", "sampleTask"

The previous example is equivalent to:

task_key = Google::Cloud::Datastore::Key.new "Task", "sampleTask"

Create an empty key:

key = datastore.key

Create an incomplete key:

key = datastore.key "User"

Create a key with a parent:

key = datastore.key [["TaskList", "default"],
                     ["Task", "sampleTask"]]
key.path #=> [["TaskList", "default"], ["Task", "sampleTask"]]

Create a key with multi-level ancestry:

key = datastore.key([
  ["User", "alice"],
  ["TaskList", "default"],
  ["Task", "sampleTask"]
])
key.path #=> [["User", "alice"], ["TaskList", "default"], [ ... ]]

Create an incomplete key with a parent:

key = datastore.key "TaskList", "default", "Task"
key.path #=> [["TaskList", "default"], ["Task", nil]]

Create a key with a project and namespace:

key = datastore.key ["TaskList", "default"], ["Task", "sampleTask"],
                    project: "my-todo-project",
                    namespace: "ns~todo-project"
key.path #=> [["TaskList", "default"], ["Task", "sampleTask"]]
key.project #=> "my-todo-project",
key.namespace #=> "ns~todo-project"

Parameters:

  • path (Array<Array(String,(String|Integer|nil))>) —

    An optional list of pairs for the key’s path. Each pair may include the key’s kind (String) and an id (Integer) or name (String). This is optional.

  • project (String) (defaults to: nil) —

    The project of the Key. This is optional.

  • namespace (String) (defaults to: nil) —

    namespace kind of the Key. This is optional.

Returns:



567
568
569
570
571
572
573
574
575
576
577
 
# File 'lib/google/cloud/datastore/dataset.rb', line 567

def key *path, project: nil, namespace: nil
  path = path.flatten.each_slice(2).to_a # group in pairs
  kind, id_or_name = path.pop
  Key.new(kind, id_or_name).tap do |k|
    k.project = project
    k.namespace = namespace
    unless path.empty?
      k.parent = key path, project: project, namespace: namespace
    end
  end
end

#project ⇒ Object

The Datastore project connected to.

Examples:

require "google/cloud/datastore"

datastore = Google::Cloud::Datastore.new(
  project: "my-todo-project",
  keyfile: "/path/to/keyfile.json"
)

datastore.project #=> "my-todo-project"


81
82
83
 
# File 'lib/google/cloud/datastore/dataset.rb', line 81

def project
  service.project
end

#query(*kinds) ⇒ Google::Cloud::Datastore::Query

Create a new Query instance. This is a convenience method to make the creation of Query objects easier.

Examples:

query = datastore.query("Task").
  where("done", "=", false)
tasks = datastore.run query

The previous example is equivalent to:

query = Google::Cloud::Datastore::Query.new.
  kind("Task").
  where("done", "=", false)
tasks = datastore.run query

Parameters:

  • kinds (String) —

    The kind of entities to query. This is optional.

Returns:



481
482
483
484
485
 
# File 'lib/google/cloud/datastore/dataset.rb', line 481

def query *kinds
  query = Query.new
  query.kind(*kinds) unless kinds.empty?
  query
end

#run(query, namespace: nil, consistency: nil) ⇒ Google::Cloud::Datastore::Dataset::QueryResults Also known as: run_query

Retrieve entities specified by a Query.

Examples:

query = datastore.query("Task").
  where("done", "=", false)
tasks = datastore.run query

Run an ancestor query with eventual consistency:

task_list_key = datastore.key "TaskList", "default"
query.kind("Task").
  ancestor(task_list_key)

tasks = datastore.run query, consistency: :eventual

Run the query within a namespace with the ‘namespace` option:

query = datastore.query("Task").
  where("done", "=", false)
tasks = datastore.run query, namespace: "ns~todo-project"

Run the query with a GQL string.

gql_query = datastore.gql "SELECT * FROM Task WHERE done = @done",
                          done: false
tasks = datastore.run gql_query

Run the GQL query within a namespace with ‘namespace` option:

gql_query = datastore.gql "SELECT * FROM Task WHERE done = @done",
                          done: false
tasks = datastore.run gql_query, namespace: "ns~todo-project"

Parameters:

Returns:



385
386
387
388
389
390
391
392
393
394
395
 
# File 'lib/google/cloud/datastore/dataset.rb', line 385

def run query, namespace: nil, consistency: nil
  ensure_service!
  unless query.is_a?(Query) || query.is_a?(GqlQuery)
    fail ArgumentError, "Cannot run a #{query.class} object."
  end
  check_consistency! consistency
  query_res = service.run_query query.to_grpc, namespace,
                                consistency: consistency
  QueryResults.from_grpc query_res, service, namespace,
                         query.to_grpc.dup
end

#save(*entities) ⇒ Array<Google::Cloud::Datastore::Entity> Also known as: upsert

Persist one or more entities to the Datastore.

Examples:

Insert a new entity:

task = datastore.entity "Task" do |t|
  t["type"] = "Personal"
  t["done"] = false
  t["priority"] = 4
  t["description"] = "Learn Cloud Datastore"
end
task.key.id #=> nil
datastore.save task
task.key.id #=> 123456

Insert multiple new entities in a batch:

task1 = datastore.entity "Task" do |t|
  t["type"] = "Personal"
  t["done"] = false
  t["priority"] = 4
  t["description"] = "Learn Cloud Datastore"
end

task2 = datastore.entity "Task" do |t|
  t["type"] = "Personal"
  t["done"] = false
  t["priority"] = 5
  t["description"] = "Integrate Cloud Datastore"
end

task_key1, task_key2 = datastore.save(task1, task2).map(&:key)

Update an existing entity:

task = datastore.find "Task", "sampleTask"
task["priority"] = 5
datastore.save task

Parameters:

  • entities (Entity) —

    One or more entity objects to be saved.

Returns:



159
160
161
 
# File 'lib/google/cloud/datastore/dataset.rb', line 159

def save *entities
  commit { |c| c.save(*entities) }
end

#transaction {|tx| ... } ⇒ Object

Creates a Datastore Transaction.

Examples:

Runs the given block in a database transaction:

require "google/cloud/datastore"

datastore = Google::Cloud::Datastore.new

task = datastore.entity "Task", "sampleTask" do |t|
  t["type"] = "Personal"
  t["done"] = false
  t["priority"] = 4
  t["description"] = "Learn Cloud Datastore"
end

datastore.transaction do |tx|
  if tx.find(task.key).nil?
    tx.save task
  end
end

If no block is given, a Transaction object is returned:

require "google/cloud/datastore"

datastore = Google::Cloud::Datastore.new

task = datastore.entity "Task", "sampleTask" do |t|
  t["type"] = "Personal"
  t["done"] = false
  t["priority"] = 4
  t["description"] = "Learn Cloud Datastore"
end

tx = datastore.transaction
begin
  if tx.find(task.key).nil?
    tx.save task
  end
  tx.commit
rescue
  tx.rollback
end

Yields:

  • (tx) —

    a block yielding a new transaction

Yield Parameters:



444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
 
# File 'lib/google/cloud/datastore/dataset.rb', line 444

def transaction
  tx = Transaction.new service
  return tx unless block_given?

  begin
    yield tx
    tx.commit
  rescue
    begin
      tx.rollback
    rescue
      raise TransactionError,
            "Transaction failed to commit and rollback."
    end
    raise TransactionError, "Transaction failed to commit."
  end
end

#update(*entities) ⇒ Array<Google::Cloud::Datastore::Entity>

Update one or more entities to the Datastore. An InvalidArgumentError will raised if the entities cannot be updated.

Examples:

Update an existing entity:

task = datastore.find "Task", "sampleTask"
task["done"] = true
datastore.save task

update multiple new entities in a batch:

query = datastore.query("Task").where("done", "=", false)
tasks = datastore.run query
tasks.each { |t| t["done"] = true }
datastore.update tasks

Parameters:

  • entities (Entity) —

    One or more entity objects to be updated.

Returns:



223
224
225
 
# File 'lib/google/cloud/datastore/dataset.rb', line 223

def update *entities
  commit { |c| c.update(*entities) }
end