Class: NVX::SDS::HttpUpload
- Inherits:
-
Object
- Object
- NVX::SDS::HttpUpload
- Defined in:
- lib/nvx/sds/httpupload.rb
Overview
Overview
The HttpUpload is used to upload files via HTTP POST. This class can upload a maximum of 256gb.
Class Method Summary collapse
-
.post_data(node, path, filename, file_data, offset, file_size, is_secure) ⇒ Object
A method to execute a nirvanix command with the parameters passed via a POST HTTP call.
-
.UploadFile(destination_path, destination_filename, local_path, overwrite, account_login, callback = nil) ⇒ Object
Overview.
Class Method Details
.post_data(node, path, filename, file_data, offset, file_size, is_secure) ⇒ Object
A method to execute a nirvanix command with the parameters passed via a POST HTTP call.
133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 |
# File 'lib/nvx/sds/httpupload.rb', line 133 def HttpUpload.post_data(node, path, filename, file_data, offset, file_size, is_secure) # setup headers including content-range to tell the upload what chunk we are uploading. boundary = "XYZBOUNDARY23487844XYZBOUNDARY1234567889" # setup the HTTP post data. content = "--" + boundary + "\r\n" + "Content-Disposition: form-data; name=\"File1\"; filename=\"#{filename}\"\r\n" + "Content-Transfer-Encoding: binary\r\n" + "Content-Type: application/octet-stream\r\n" + "Content-Range: #{offset}-#{offset + file_data.length - 1}/#{file_size}\r\n" + "\r\n" + "#{file_data}\r\n" + "--" + boundary + "--\r\n" # pass headers including Content-Range to define the chunk that is being sent. headers = { 'Content-Type' => "multipart/form-data; boundary=#{boundary}", 'User-Agent' => USERAGENT } # set the port based on the is_secure flag. port = is_secure ? 443 : 80 # Create the first http(s) object. if @h.nil? @h = Net::HTTP.new(node, port).start end # if you are just testing and do not have a SSL certificate setup properly you can # disable warnings with the below line. Just uncomment to ignore SSL session warnings. @h.verify_mode = OpenSSL::SSL::VERIFY_NONE if is_secure # put the SSL in the correct mode based on the apicommand @h.use_ssl = is_secure # do the upload and return the response. req = Net::HTTP::Post.new(path, headers) req.content_length = content.length req.content_type = "multipart/form-data; boundary=#{boundary}" req.body = content response = @h.request(req) #print "\r\nContent: " + content + "\r\n\r\n" #print "RESPONSE XML: " + response.body + "\r\n\r\n" # read the xml document and get any errors that are returned. doc = REXML::Document.new(response.body) response_code = (text = doc.root.elements["//Response/ResponseCode"].get_text and text.value) if response_code.to_i == 70121 raise RetryException.new end if response_code.to_i != 0 = (text = doc.root.elements["//Response/ErrorMessage"].get_text and text.value) raise end return doc end |
.UploadFile(destination_path, destination_filename, local_path, overwrite, account_login, callback = nil) ⇒ Object
Overview
The HttpUpload.UploadFile is used to upload files via HTTP POST. This class can upload a maximum of 256gb.
Parameters
destination_path
The path at Nirvanix where the file will be uploaded
destination_filename
The remote filename
local_path
The path to the file that will be uploaded on the local file system.
overwrite
Determines if the upload should overwrite the file at Nirvanix if it exists.
account_login
the account information from the Session.new that was created with valid login information.
callback
The callback is an object with two methods for recieving updates while the download is in progress.
def percent(percent, bytes_uploaded)
print "Percent: #{percent} - Bytes: #{bytes_uploaded}\r\n"
end
Returns a the percentage complete along with the number of bytes uploaded so far.
def warning()
print "Warning: #{}\r\n"
end
Returns a warning message when there is an internal connectivity issue with the upload.
67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 |
# File 'lib/nvx/sds/httpupload.rb', line 67 def HttpUpload.UploadFile(destination_path, destination_filename, local_path, overwrite, account_login, callback = nil) # Get the Upload node passing in the total file size. file_size = File.size(local_path) params = Array.new params << APIParam.new("sizeBytes", file_size) params << APIParam.new("destFolderPath", destination_path) params << APIParam.new("fileOverwrite", overwrite) result = Transport.execute_command_post(APICommand.GetStorageNodeExtended, params, account_login) # extract the upload token, host name and build a new transfer object. upload_token = result.root.elements["//UploadToken"].get_text.value node = result.root.elements["//UploadHost"].get_text.value # set the URL based on the node that was returned from Nirvanix. # Open the local file file = File.open(local_path, "rb") offset = 0 retry_count = 0 path = "/upload.ashx?uploadToken=#{upload_token}&destFolderPath=#{destination_path}" percent_uploaded = 0 # Loop through the entire file uploading each file part. while !file.eof? # read a chunk of data from the file. file_data = file.read(BLOCKSIZE) # Send a chunk to Nirvanix using the upload token and node. begin tmppath = path # adjust the path to include rangeOverwrite when trying to write to a previous offset. if retry_count > 0 tmppath = path + "&rangeOverwrite=true" end params = post_data(node, tmppath, destination_filename, file_data, offset, file_size, false) # the SystemCallError handles ETIMEOUT, EPIPE, EHOSTDOWN for timeouts and broken pipes. rescue SocketError, SystemCallError, IOError, RetryException # set the file position to the previous offset to retry that block. file.pos = offset retry_count += 1 # if the maximum retry count has been reached raise an exception to the outside world. if retry_count == 10 raise RetryException.new("Connection failure at offset: #{offset}") end # If the callback object is available pass a warning specifying the retry count. if !callback.nil? callback.warning("Connection failure at offset: #{offset} Retry count: #{retry_count}") end redo end # Reset the retry count to 0 since this was transmission was successful. retry_count = 0 # advance offset based on how much data was read. offset += file_data.length # get percentage complete new_percentage = (offset * 100) / file_size # if the percentage has changed update the callback. if new_percentage > percent_uploaded percent_uploaded = new_percentage if !callback.nil? callback.percent(percent_uploaded, offset) end end end end |