Class: IOStreams::Paths::SFTP

Inherits:
IOStreams::Path show all
Includes:
SemanticLogger::Loggable
Defined in:
lib/io_streams/paths/sftp.rb

Overview

Read a file from a remote sftp server.

Example:

IOStreams.
  path("sftp://example.org/path/file.txt", username: "jbloggs", password: "secret", compression: false).
  reader do |input|
    puts input.read
  end

Note:

  • raises Net::SFTP::StatusException when the file could not be read.

Write to a file on a remote sftp server.

Example:

IOStreams.
  path("sftp://example.org/path/file.txt", username: "jbloggs", password: "secret", compression: false).
  writer do |output|
    output.write('Hello World')
  end

Class Attribute Summary collapse

Instance Attribute Summary collapse

Attributes inherited from IOStreams::Path

#path

Attributes inherited from Stream

#io_stream

Instance Method Summary collapse

Methods inherited from IOStreams::Path

#<=>, #==, #absolute?, #children, #compressed?, #copy_from, #delete, #delete_all, #directory, #encrypted?, #exist?, #inspect, #join, #mkpath, #move_to, #partial_files_visible?, #realpath, #size

Methods inherited from Stream

#basename, #copy_from, #copy_to, #dirname, #each, #extension, #extname, #file_name, #file_name=, #format, #format=, #format_options, #format_options=, #option, #option_or_stream, #pipeline, #read, #reader, #setting, #stream, #write, #writer

Constructor Details

#initialize(url, username: nil, password: nil, ssh_options: {}) ⇒ SFTP

Stream to a remote file over sftp.

url: [String]

"sftp://<host_name>/<file_name>"

username: [String]

Name of user to login with.

password: [String]

Password for the user.

ssh_options: [Hash]

- IdentityKey [String]
  The identity key that this client should use to talk to this host.
  Under the covers this value is written to a file and then the file name is passed as `IdentityFile`
- HostKey [String]
  The expected SSH Host key that is presented by the remote host.
  Instead of storing the host key in the `known_hosts` file, it can be supplied explicity
  using this option.
  Under the covers this value is written to a file and then the file name is passed as `UserKnownHostsFile`
  Notes:
  - It must contain the entire line that would be stored in `known_hosts`,
    including the hostname, ip address, key type and key value. This value is written as-is into a
    "known_hosts" like file and then passed into sftp using the `UserKnownHostsFile` option.
  - The easiest way to generate the required is to use `ssh-keyscan` and then supply that value in this field.
    For example: `ssh-keyscan hostname`
- Any other options supported by ssh_config.
  `man ssh_config` to see all available options.

Examples:

# Display the contents of a remote file
IOStreams.path("sftp://test.com/path/file_name.csv", username: "jack", password: "OpenSesame").reader do |io|
  puts io.read
end

# Full url showing all the optional elements that can be set via the url:
sftp://username:password@hostname:22/path/file_name

# Display the contents of a remote file, supplying the username and password in the url:
IOStreams.path("sftp://jack:[email protected]:22/path/file_name.csv").reader do |io|
  puts io.read
end

# Display the contents of a remote file, supplying the username and password as arguments:
IOStreams.path("sftp://test.com/path/file_name.csv", username: "jack", password: "OpenSesame").reader do |io|
  puts io.read
end

# When using the sftp executable use an identity file instead of a password to authenticate:
IOStreams.path("sftp://test.com/path/file_name.csv",
               username:    "jack",
               ssh_options: {IdentityFile: "~/.ssh/private_key"}).reader do |io|
  puts io.read
end

Raises:

  • (ArgumentError) 


94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
 
# File 'lib/io_streams/paths/sftp.rb', line 94

def initialize(url, username: nil, password: nil, ssh_options: {})
  uri = Utils::URI.new(url)
  raise(ArgumentError, "Invalid URL. Required Format: 'sftp://<host_name>/<file_name>'") unless uri.scheme == "sftp"

  @hostname = uri.hostname
  @mkdir    = false
  @username = username || uri.user
  @url      = url
  @password = password || uri.password
  @port     = uri.port || 22
  # Not Ruby 2.5 yet: transform_keys(&:to_s)
  @ssh_options = {}
  ssh_options.each_pair { |key, value| @ssh_options[key.to_s] = value }
  @ssh_options.merge(uri.query) if uri.query

  super(uri.path)
end

Class Attribute Details

.before_password_wait_seconds ⇒ Object

Returns the value of attribute before_password_wait_seconds.



29
30
31
 
# File 'lib/io_streams/paths/sftp.rb', line 29

def before_password_wait_seconds
  @before_password_wait_seconds
end

.sftp_bin ⇒ Object

Returns the value of attribute sftp_bin.



29
30
31
 
# File 'lib/io_streams/paths/sftp.rb', line 29

def sftp_bin
  @sftp_bin
end

.sshpass_bin ⇒ Object

Returns the value of attribute sshpass_bin.



29
30
31
 
# File 'lib/io_streams/paths/sftp.rb', line 29

def sshpass_bin
  @sshpass_bin
end

.sshpass_wait_seconds ⇒ Object

Returns the value of attribute sshpass_wait_seconds.



29
30
31
 
# File 'lib/io_streams/paths/sftp.rb', line 29

def sshpass_wait_seconds
  @sshpass_wait_seconds
end

Instance Attribute Details

#hostname ⇒ Object (readonly)

Returns the value of attribute hostname.



37
38
39
 
# File 'lib/io_streams/paths/sftp.rb', line 37

def hostname
  @hostname
end

#port ⇒ Object (readonly)

Returns the value of attribute port.



37
38
39
 
# File 'lib/io_streams/paths/sftp.rb', line 37

def port
  @port
end

#ssh_options ⇒ Object (readonly)

Returns the value of attribute ssh_options.



37
38
39
 
# File 'lib/io_streams/paths/sftp.rb', line 37

def ssh_options
  @ssh_options
end

#url ⇒ Object (readonly)

Returns the value of attribute url.



37
38
39
 
# File 'lib/io_streams/paths/sftp.rb', line 37

def url
  @url
end

#username ⇒ Object (readonly)

Returns the value of attribute username.



37
38
39
 
# File 'lib/io_streams/paths/sftp.rb', line 37

def username
  @username
end

Instance Method Details

#each_child(pattern = "*", case_sensitive: true, directories: false, hidden: false) ⇒ Object

Search for files on the remote sftp server that match the provided pattern.

The pattern matching works like Net::SFTP::Operations::Dir.glob and Dir.glob Each child also returns attributes that contain the file size, ownership, file dates and other details.

Example Code: IOStreams.

path("sftp://sftp.example.org/my_files", username: username, password: password).
each_child('**/*.{csv,txt}') do |input, attributes|
  puts "#{input.to_s} #{attributes}"
end

Example Output: ssftp.example.org/a/b/c/test.txt :size=>37, :owner=>“test_owner”, :group=>“test_group”,

:permissions=>420, :atime=>1572378136, :mtime=>1572378136, :link_count=>1, :extended=>{}



144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
 
# File 'lib/io_streams/paths/sftp.rb', line 144

def each_child(pattern = "*", case_sensitive: true, directories: false, hidden: false)
  Utils.load_soft_dependency("net-sftp", "SFTP glob capability", "net/sftp") unless defined?(Net::SFTP)

  flags = ::File::FNM_EXTGLOB
  flags |= ::File::FNM_CASEFOLD unless case_sensitive
  flags |= ::File::FNM_DOTMATCH if hidden

  Net::SFTP.start(hostname, username, build_ssh_options) do |sftp|
    sftp.dir.glob(".", pattern, flags) do |path|
      next if !directories && !path.file?

      new_path = self.class.new("sftp://#{hostname}/#{path.name}", username: username, password: password, **ssh_options)
      yield(new_path, path.attributes.attributes)
    end
  end
  nil
end

#mkdir ⇒ Object

Note that mkdir is delayed and only executed when the file write is performed.



122
123
124
125
 
# File 'lib/io_streams/paths/sftp.rb', line 122

def mkdir
  @mkdir = true
  self
end

#relative? ⇒ Boolean

Does not support relative file names since there is no concept of current working directory

Returns:

  • (Boolean) 


113
114
115
 
# File 'lib/io_streams/paths/sftp.rb', line 113

def relative?
  false
end

#to_s ⇒ Object 



117
118
119
 
# File 'lib/io_streams/paths/sftp.rb', line 117

def to_s
  url
end