Asperalm - Laurent Aspera CLI and Ruby library

Laurent/2016-2018

Version : 0.6.5

This Gem provides a Ruby language interface to FASP session start and a cross platform (ruby based) command line tool (CLI) using (mostly REST) Aspera Products APIs for :

  • Server
  • Node
  • Shares
  • Faspex
  • Console
  • Orchestrator
  • Files
  • ATS
  • and more...

Disclaimer: This GEM is not endorsed/supported by IBM/Aspera, use at your risk, and not in production environments.

That being said, it is very powerful and gets things done, it's also a great tool to learn Aspera APIs.

Overview

This is a Ruby Gem that provides the following features:

  • Supports most Aspera server products (on-premise and SaaS)
  • Configuration file for products URLs and credentials (and any parameter)
  • FASP transfer agent can be: FaspManager (local ascp), or Connect Client, or a transfer node
  • Transfer parameters can be altered by modification of transferspec, this includes requiring multi-session transfer on nodes
  • Allows transfers from products to products, essentially at node level
  • Supports FaspStream creation (using Node API)
  • Additional command plugins can be written by the user
  • Parameter values can be passed on command line, in configuration file, in env var, in files
  • Parameter values and command can be provided in short format (must be unique)
  • Supports download of faspex "external" links
  • Supports "legacy" ssh based FASP transfers and remote commands (ascmd)
  • A FASPManager class for Ruby
  • REST and OAuth classes for use with Aspera products APIs
  • A command line tool: aslmcli

Ruby has been chosen as language as it is used in most Aspera products (consistency), and the interpret can be found for most platforms (portability).

This gem is provided as-is, and is not intended to be a complete CLI, or industry-grade product. This is a sample. Aspera provides a CLI tool here: http://downloads.asperasoft.com/en/downloads/62 .

The CLI's folder where configuration and cache files are kept is $HOME/.aspera/aslmcli

Requires Ruby 2.4+, should work with 2.0+.

In examples below, command line operations (starting with $) are shown using a standard shell: bash.

Quick Start

Pre-requisite : Ruby

General

aslmcli requires a ruby interpreter with priviledge to install gems. Refer to the following sections for specific operating systems.

MacOS X

Ruby comes pre-installed on MacOSx. Nevertheless, installing new gems require admin privilege (sudo) and the version is not up to date.

It is better to install "homebrew", from here: https://brew.sh/, and then install Ruby:

$ brew install ruby

Windows

On windows you can get it from here: https://rubyinstaller.org/ .

Linux

$ yum install ruby rubygems

Note that Ruby 2+ is required, if you have an older Linux (e.g. CentOS 6), you should install "rvm" https://rvm.io/ and install a newer Ruby.

Installation

Once you have ruby and rights to install gems: Install the gem and its dependencies:

$ gem install asperalm

The tool can be used right away: aslmcli

Installation of the FASP protocol

For any FASP based file transfer, the FASP protocol and a valid license id required. When the server side is "connect" enabled, one can the the connect client license. The connect client can be installed by visiting the page: http://downloads.asperasoft.com/connect2/

Alternatively, the connect client can be downloaded using the aslmcli, see section: Download FASP.

Configuration for Aspera Demo Server

One can test the "server" application using the well known demo server:

$ aslmcli config id aspera_demo_server update --url=ssh://demo.asperasoft.com:33001 --username=asperaweb --password=demoaspera
$ aslmcli config id default set server aspera_demo_server 
$ aslmcli server browse /aspera-test-dir-large
$ aslmcli server download /aspera-test-dir-large/200MB

This creates a parameter set "aspera_demo_server" and set it as default for application "server"

Configuration for Applications

For Faspex, Shares, Node (including ATS, Aspera Transfer Service), Console, only username/password and url are required (either on command line, or from config file). Those can usually be provided on the command line:

$ aslmcli shares repo browse / --url=https://10.25.0.6 --username=john --password=4sp3ra 

This can also be provisioned in a config file:

1$ aslmcli config id shares06 set url https://10.25.0.6
2$ aslmcli config id shares06 set username john
3$ aslmcli config id shares06 set password 4sp3ra
4$ aslmcli config id default set shares shares06 
5$ aslmcli config show
6$ aslmcli shares repo browse /

The three first commands build a parameter set. Note that this can also be done with one single command:

$ aslmcli config id shares06 init @json:'{"url":"https://10.25.0.6","username":"john","password":"4sp3ra"}'

The fourth command defines this parameter set as the default parameter set for the specified application ("shares"). The 5th command dumps the configuration file. Alternative parameter sets can be used with option "-P<parameter set>" (or --load-params=<parameter set>)

Eventually, the last command shows a call to the shares application using default parameters.

Configuration for Aspera Files

Aspera Files APIs does not support Basic HTTP authentication (see section "Authentication"). Using the CLI with Aspera Files will require the generation of a "Bearer token", this is can be done by authentication with a web interface (see section "Graphical interactions").

A more convenient way to use the CLI with Aspera Files, a possibility is to do the following (JWT auth):

  • Create a private/public key pair, as specified in section: "Private/Public Keys"

  • Register a new application in the Aspera Files Admin GUI (refer to section "Authentication"). Here, use the contents of this file (generated in step 2) for public key: $HOME/.aspera/aslmcli/filesapikey.pub

  • set your Aspera Files default parameters:

$ aslmcli config id default set files files_myorg
$ aslmcli config id files_myorg set url https://myorg.asperafiles.com
$ aslmcli config id files_myorg set client_id MyClIeNtKeY
$ aslmcli config id files_myorg set client_secret MyClIeNtSeCrEtMyClIeNtSeCrEt
$ aslmcli config id files_myorg set username [email protected]
$ aslmcli config id files_myorg set private_key @val:@file:"$HOME/.aspera/aslmcli/filesapikey"

Note: the private key argument represents the actual PEM string. In order to read the content from a file, use the @file: prefix. But if the @file: argument is used as is, it will read the file and set in the config file. So to keep the "@file" tag in the configuration file, the @val: prefix is added.

  • CLI is ready to use:
$ aslmcli files repo browse /
:..............................:........:................:...........:......................:..............:
:             name             :  type  : recursive_size :   size    :    modified_time     : access_level :
:..............................:........:................:...........:......................:..............:
: Summer 2016 Training         : link   :                :           : 2016-07-25T15:21:22Z : edit         :
: Laurent Garage SE            : folder : 19316893       :           :                      : edit         :
: Italy Training               : folder : 312068540      :           :                      : edit         :
: Cheese pile.jpeg             : file   :                : 9824      : 2016-11-16T12:10:25Z : edit         :
: Aspera Video                 : folder : 122237276      :           :                      : edit         :
:..............................:........:................:...........:......................:..............:

Plugins and applications

The CLI tool uses a plugin mechanism. The first level command (just after aslmcli on the command line) is the name of the concerned plugin which will execute the command. Each plugin usually represent commands sent to a specific application. For instance, the plugin "faspex" allows operations on the application "Aspera Faspex".

Options and Arguments

Command options and arguments are normally provided on command line, i.e. aslmcli command --option-name=VAL1 VAL2 takes "VAL1" as the value for option option_name, and "VAL2" as the value for first argument of command: command.

It is also possible to retrieve a value (option and argument) with the following special prefixes:

  • @val:VALUE , just take the specified value, e.g. [email protected]:laurent, allows for meta characters
  • @file:PATH , read value from a file (prefix "~/" is replaced with $HOME), e.g. [email protected]:$HOME/.ssh/mykey
  • @env:ENVVAR , read from a named env var, [email protected]:MYPASSVAR

In addition it is possible to decode some values by prefixing :

  • @base64: to decode base64 encoded string
  • @json: to decode JSON values
  • @zlib: to uncompress data

Example:

@zlib:@base64:@file:myfile.dat

This will read the content of the specified file, then, base64 decode, then unzip.

The special option "--" stop option processing, so following values are taken as arguments.

Environment variable starting with prefix: ASLMCLI_ are taken as option values, i.e. ASLMCLI_OPTION_NAME is for --option-name.

Some ptions have hardcoded default values. All options can be defined in a configuration file (see Configuration file). Options values can be displays for a given command by providing the --show-config option.

Configuration file

The use of the configuration file is not mandatory. All parameters can be set on command line. A configuration file provides a way to define default values, especially for authentication parameters, thus avoiding to always have to specify those parameters on the command line.

The default configuration file is: $HOME/.aspera/aslmcli/config.yaml (can be overriden with option --config-file). It is composed with "parameter sets" containing "parameters". (The first level is an associative array whose values are associative arrays or parameters).

A good practice is to not manually edit this file and use modification commands. If necessary, the configuration file can be edited with:

$ aslmcli config open

The configuration file can be modified using the commands:

aslmcli config id <parameter set name> set|delete|show|initialize|update

The command update allows the easy creation of parameter set by simply providing the options in their normal format, e.g. :

aslmcli config id node_to_node update --url=https://10.25.0.4:9092 --username=node_user --password=node_pass [email protected]:'{"precalculate_job_size":true}' --transfer=node [email protected]:'{"url":"https://10.25.0.8:9092","username":"node_user2","password":"node_pass2"}'

Two parameter sets are reserved:

  • "config" is reserved for the global parameters of the aslmcli tool. It contains a special parameter: "version" showing the CLI version used to create this file. It is used to check compatibility.
  • "default" is reserved to define the default parameter set name used for plugins.

Usually, a configuration "parameter set" corresponds to a set of parameters used for a specific plugin.

The configuration file is a hash in a YAML file. Example:

config:
  version: 0.3.7
  loglevel: debug
default:
  faspex: myfaspparams
myfaspparams:
  url: https://faspex.my.org/aspera/faspex
  username: admin
  password: MyPassword
  • The configuration was created with CLI version 0.3.7
  • the log level is set to "debug"
  • the default parameter set to load for plugin "faspex" is : myfaspparams
  • the parameter set "myfaspparams" defines some parameters: the URL and credentials

A configuration file is located:

  • if a parameter --config-file is specified, this defines the configuration file to use
  • else the default file is used: $HOME/.aspera/aslmcli/config.yaml

Default parameters are loaded using this algorithm:

  • if option '--load-params=xxxx' is specified (or -Pxxxx), this reads the parameter set specified
    • else if option --no-default is specified, then dont load default
    • else it looks for the name of the default parameter set in section "default" and loads it
  • environment variables are evaluated
  • command line options are evaluated

A parameter value can be specified with the same syntax as section "Option and Argument values"

Parameters are evaluated in the order of command line.

To avoid loading the default parameter set for a plugin, just specify a non existing configuration: -Pnone

On command line, words in parameter names are separated by a dash, in configuration file, separator is an underscore. E.g. --transfer-name on command line gives transfer_node in configuration file.

Note: before version 0.4.5, some keys could be ruby symbols, from 0.4.5 all keys are strings. To convert olver versions, remove the leading ":" in fronty of keys.

TODO: interactive input

options -T -A

Learning Aspera Product APIs (REST)

This CLI uses REST APIs. To display HTTP calls, use argument -r or --rest-debug, this is useful to display exact content or HTTP requests and responses.

In order to get traces of execution, use argument : --log-level=debug

Authentication

Aspera Faspex / Shares / Console / Node

Only Basic authentication is supported. A "username" and "password" are provided, either on command line (--username, --password) or in the configuration file.

Aspera Files

Aspera Files supports a more powerful and secure authentication mechanism: Oauth. HTTP Basic authentication is not supported (deprecated).

With OAuth, the application (aslmcli) must be identified, and a valid Aspera Files user must be used to access Aspera Files. Then a "Bearer" token is used for HTTP authentication.

First the application (aslmcli) must be declared in the Files GUI (see https://aspera.asperafiles.com/helpcenter/admin/organization/registering-an-api-client ). By declaring the application, a "client_id" and "client_secret" are generated:

Files-admin-organization-apiclient-clientdetails

It is possible to use the Aspera Files API, but a web browser is required to generate the token. aslmcli will either display the URL to be entered in a local browser, or open a browser directly (various options are proposed).

It is also possible to enable browser-less authentication by using JWT, in this case a private/public key pair is required (see section: Generating a key pair), the public key is provided to Aspera Files:

Files-admin-organization-apiclient-authoptions

Upon successful authentication, auth token are saved (cache) in local files, and can be used subsequently. Expired token can be refreshed.

Graphical interactions: Browser and Text Editor

Some actions may require the use of a graphical tool:

  • a browser for Aspera Files authentication
  • a text editor for configuration file edition

By default the CLI will assume that a graphical environment is available on windows, and on other systems, rely on the presence of the "DISPLAY" environment variable. It is also possible to force the graphical mode with option --gui-mode :

  • --gui-mode=graphical forces a graphical environment, a browser will be opened for URLs or a text editor for file edition.
  • --gui-mode=text forces a text environment, the URL or file path to open is displayed on terminal.

Private/Public Keys

In order to use JWT for Aspera Files API client authentication, a private/public key pair must be generated.

For example, generate a passphrase-less keypair with ssh-keygen:

$ ssh-keygen -t rsa -f ~/.aspera/aslmcli/filesapikey -N ''

One can also use the "openssl" utility: (on some openssl implementation there is option: -nodes (no DES))

$ APIKEY=~/.aspera/aslmcli/filesapikey
$ openssl genrsa -passout pass:dummypassword -out ${APIKEY}.protected 2048
$ openssl rsa -passin pass:dummypassword -in ${APIKEY}.protected -out ${APIKEY}
$ openssl rsa -pubout -in ${APIKEY} -out ${APIKEY}.pub
$ rm -f ${APIKEY}.protected

Or simply use the CLI:

$ aslmcli config genkey ~/.aspera/aslmcli/filesapikey

FASP agents

The CLI provides access to Aspera Applications functions through REST APIs, it also allows FASP based transfers (upload and download).

Any FASP parameter can be set by changing parameters in the associated "transfer spec". The CLI standadizes on the use of "transfer spec" and does not support directly ascp options. It is nevertheless possible to add ascp options (for fasp manager only, but not node api or connect) using the special transfer spec parameter: EX_ascp_args.

Three Transfer agents are currently supported to start transfers :

FASPManager API based (command line)

By default the CLI will use the Aspera Connect Client FASP part, in this case it requires the installation of the Aspera Connect Client to be able to execute FASP based transfers. The CLI will try to automatically locate the Aspera Protocol (ascp). This is option: --transfer=ascp. Note that parameters are always provided with a "transfer spec".

Aspera Connect Client GUI

By specifying option: --transfer=connect, the CLI will start transfers in the Aspera Connect Client.

Aspera Node API : Node to node transfers

By specifying option: --transfer=node, the CLI will start transfers in an Aspera Transfer Server using the Node API.

If a default node has been configured in the configuration file, then this node is used by default else the parameter --transfer-node is required. The node specification shall be a hash table with three keys: url, username and password, corresponding to the URL of the node API and associated credentials (node user or access key).

The --transfer-node parameter can directly specify a pre-configured parameter set : [email protected]:<psetname> or specified using the option syntax : [email protected]:'{"url":"https://...","username":"theuser","password":"thepass"}'

Transfer spec

The "Transfer spec" specifies FASP protocol session startup parameters.

Destination folder for transfers

Use parameter --to-folder=dst_path to set destination folder on download or upload. By default destination is "." for downloads, and "/" for uploads. Note that it is equivalent to setting "destination_root" in transfer spec using option [email protected]:'"destination_root":"<em>dst_path</em>"'

Examples

Transfer parameters are contained in a "transfer spec", an associative array. Changing a transfer spec is done by providing parameters in a JSON syntax. For instance to override the FASP SSH port to a specific TCP port:

[email protected]:'{"ssh_port":12345}'

To force http fallback mode:

[email protected]:'{"http_fallback":"force"}'

In Node based transfers, Multi-session is available, simply add [email protected]:'{...}':

[email protected]:'{"multi_session":10,"multi_session_threshold":1,"target_rate_kbps":500000,"checksum_type":"none","cookie":"custom:aslmcli:Laurent:My Transfer"}'

Parameter list

All standard transfer spec parameter can be overloaded. To display parameter, run in debug mode (--log-level=debug). Transfer spec can also be saved/overridden in the config file.

(UNDER CONSTRUCTION ref)

  • F=Fasp Manager(local FASP execution)
  • N=remote node(node API)
  • C=Connect Client(web plugin)

Req/Def : Required or default value (- means emty)

Fields with EX_ prefix are specific to aslmcli in local mode.

arg: related ascp argument or env var suffix (PASS for ASPERA_SCP_PASS)

FieldReq/DefTypeFNCargDescription
directionRequiredstringYYY--modeDirection: "send" or "receive"
remote_hostRequiredstringYYY--hostIP or fully qualified domain name of the remote server
remote_userRequiredstringYYY--userRemote user. Default value is "xfer" on node or connect.
destination_rootRequiredstringYYYlast argDestination root directory.
title-stringNYY-Title of the transfer
tags-hashYYY--tags
--tags64
Metadata for transfer
token-stringYYYTOKEN
-W
Authorization token: Bearer, Basic or ATM
cookie-stringYYYCOOKIEMetadata for transfer (older,string)
remote_access_keyTODOstring?Node only?
source_root-string--source-prefix
--source-prefix64
Source root directory.(TODO: verify option)
fasp_port33001integerYYY-OSpecifies fasp (UDP) port.
ssh_port22 or 33001integerYYY-PSpecifies ssh (TCP) port.
rate_policyserver configstringYYY--policyValid literals include "low","fair","high" and "fixed".
symlink_policyfollowstringYYY--symbolic-linkscopy, follow, copy+force, skip. Default is follow. Handle source side symbolic links by following the link (follow), copying the link itself (copy), skipping (skip), or forcibly copying the link itself (copy+force).
target_rate_kbps-integerYYY-lSpecifies desired speed for the transfer.
min_rate_kbps0integerYYY-mSet the minimum transfer rate in kilobits per second.
ciphernonestringYYY-cin transit encryption type.
none, aes-128, aes-256
content_protectionencrypt
decrypt
string--file-crypt=encryption at rest
content_protection_password-stringPASSSpecifies a string password.
overwritediffstringYYY--overwriteOverwrite destination files with the source files of the same name.
never, always, diff, older, or diff+older
retry_durationstringTODOSpecifies how long to wait before retrying transfer. (e.g. "5min")
http_fallbackbool (node), integer-y
TODO
When true(1), attempts to perform an HTTP transfer if a fasp transfer cannot be performed.
create_dirbooleanYYY-dSpecifies whether to create new directories.
precalculate_job_sizesrv. def.booleanYYY--precalculate-job-sizeSpecifies whether to precalculate the job size.
delete_sourcebooleanY??
remove_after_transferbooleanY?Specifies whether to remove file after transfer.
remove_empty_directoriesbooleanY?Specifies whether to remove empty directories.
multi_session1integerNYN-CSpecifies how many parts the transfer is in.
multi_session_thresholdnullintegerNYN-in bytes
dgram_sizeintegerY-Zin bytes
compressionintegerascp4 only, 0 / 1?
read_threadsinteger-ascp4 only
write_threadsinteger-ascp4 only
use_ascp4falsebooleanY-specify version of protocol
pathssource files (dest)arraypositional
--file-list
--file-pair-list
Contains a path to the source (required) and a path to the destination.
http_fallback_portintegerY-tSpecifies http port.
https_fallback_portintegertodoSpecifies https port.
cipher_allowedstring-returned by node API. Valid literals include "aes-128" and "none".
target_rate_cap_kbpsN??-Returned by upload/download_setup node api.
rate_policy_allowed-returned by node API. Specifies most aggressive rate policy that is allowed. Valid literals include "low", "fair","high" and "fixed".
ssh_private_keystring-todo
remote_password-stringYYYPASSSSH session password
resume_policyfaspmgr:
none
other:
sparse_csum
stringYYY-knone,attrs,sparse_csum,full_csum
authenticationfaspmgr:
none
other:
sparse_csum
stringYYN-token: Aspera web keys are provided to allow transparent web based session initiation. on connect: password is not asked. Else, password is asked, and keys are not provided.
EX_ssh_key_value-stringYNNKEYPrivate key used for SSH authentication
EX_ssh_key_paths-arrayYNN-iUse public key authentication and specify the private key file
EX_fallback_key-stringYNN-YThe HTTPS transfer's key file name
EX_fallback_cert-stringYNN-IThe HTTPS certificate's file name
EX_at_rest_password-stringYNNFILEPASSPassphrase used for at rest encryption or decryption
EX_proxy_password-stringYNNPROXY_PASSTODO
EX_quiet-booleanYNN-qQuiet flag, disable progress display
EX_fasp_proxy_url-stringYNN--proxySpecify the address of the Aspera high-speed proxy server
EX_http_proxy_url-stringYNN-xSpecify the proxy server address used by HTTP Fallback
EX_ascp_args-arrayYNNsameAdd command line arguments to ascp
EX_http_transfer_jpeg0integerYNN-jHTTP transfers as JPEG file

FASP Stream

It is possible to start a FASPStream session using the node API:

Use the "node stream create" command, then arguments are provided as a "transfer spec".

./bin/aslmcli node stream create [email protected]:'{"direction":"send","source":"udp://233.3.3.4:3000?loopback=1&ttl=2","destination":"udp://233.3.3.3:3001/","remote_host":"localhost","remote_user":"stream","remote_password":"XXXX"}' --load-params=stream

Watchfolder

Refer to Aspera documentation for watch folder creation.

aslmcli supports remote operations through the node API. Operations are:

  • Start watchd and watchfolderd services running as a system user having access to files
  • configure a watchfolder to define automated transfers
$ aslmcli node service create @json:'{"id":"mywatchd","type":"WATCHD","run_as":{"user":"user1"}}'
$ aslmcli node service create @json:'{"id":"mywatchfolderd","type":"WATCHFOLDERD","run_as":{"user":"user1"}}'
$ aslmcli node watch_folder create @json:'{"id":"mywfolder","source_dir":"/watch1","target_dir":"/","transport":{"host":"10.25.0.4","user":"user1","pass":"Aspera123_"}}'

Download FASP

The CLI allows download of the FASP protocol in connect client :

$ ./bin/aslmcli client connect list
:...............................................:......................................:..............:
:                      id                       :                title                 :   version    :
:...............................................:......................................:..............:
: urn:uuid:589F9EE5-0489-4F73-9982-A612FAC70C4E : Aspera Connect for Windows           : 3.7.0.138427 :
: urn:uuid:A3820D20-083E-11E2-892E-0800200C9A66 : Aspera Connect for Windows 64-bit    : 3.7.0.138427 :
: urn:uuid:589F9EE5-0489-4F73-9982-A612FAC70C4E : Aspera Connect for Windows XP        : 3.7.0.138427 :
: urn:uuid:55425020-083E-11E2-892E-0800200C9A66 : Aspera Connect for Windows XP 64-bit : 3.7.0.138427 :
: urn:uuid:D8629AD2-6898-4811-A46F-2AF386531BFF : Aspera Connect for Mac Intel 10.6    : 3.6.1.111259 :
: urn:uuid:D8629AD2-6898-4811-A46F-2AF386531BFF : Aspera Connect for Mac Intel         : 3.7.0.138427 :
: urn:uuid:213C9370-22B1-11E2-81C1-0800200C9A66 : Aspera Connect for Linux 32          : 3.6.2.117442 :
: urn:uuid:97F94DF0-22B1-11E2-81C1-0800200C9A66 : Aspera Connect for Linux 64          : 3.7.2.141527 :
:...............................................:......................................:..............:
$ aslmcli client connect id 'Aspera Connect for Mac Intel 10.6' links list
:.............................................:..........................:.......................................................................:..........:...............:
:                    title                    :           type           :                                 href                                  : hreflang :      rel      :
:.............................................:..........................:.......................................................................:..........:...............:
: Mac Intel Installer                         : application/octet-stream : bin/AsperaConnect-3.6.1.111259-mac-intel-10.6.dmg                     : en       : enclosure     :
: Aspera Connect for Mac HTML Documentation   : text/html                :                                                                       : en       : documentation :
: Aspera Connect PDF Documentation for Mac OS : application/pdf          : docs/user/osx/ja-jp/pdf/Connect_User_3.7.0_OSX_ja-jp.pdf              : ja-jp    : documentation :
: Aspera Connect PDF Documentation for Mac OS : application/pdf          : docs/user/osx/en/pdf/Connect_User_3.7.0_OSX.pdf                       : en       : documentation :
: Aspera Connect PDF Documentation for Mac OS : application/pdf          : docs/user/osx/es-es/pdf/Connect_User_3.7.0_OSX_es-es.pdf              : es-es    : documentation :
: Aspera Connect PDF Documentation for Mac OS : application/pdf          : docs/user/osx/fr-fr/pdf/Connect_User_3.7.0_OSX_fr-fr.pdf              : fr-fr    : documentation :
: Aspera Connect PDF Documentation for Mac OS : application/pdf          : docs/user/osx/zh-cn/pdf/Connect_User_3.7.0_OSX_zh-cn.pdf              : zh-cn    : documentation :
: Aspera Connect for Mac Release Notes        : text/html                : http://www.asperasoft.com/en/release_notes/default_1/release_notes_54 : en       : release-notes :
:.............................................:..........................:.......................................................................:..........:...............:
$ aslmcli client connect id 'Aspera Connect for Mac Intel 10.6' links id 'Mac Intel Installer' download --to-folder=.
downloaded: AsperaConnect-3.6.1.111259-mac-intel-10.6.dmg

Transfer filter

special in node:

"transfer_filter"=>"t['status'].eql?('completed') and t['start_spec']['remote_user'].eql?('faspex')", :file_filter=>"f['status'].eql?('completed') and 0 != f['size'] and t['start_spec']['direction'].eql?('send')"

Examples

SHOD to ATS

Access to a "Shares on Demand" (SHOD) server on AWS is provided by a partner. And we need to transfer files from this third party SHOD instance into our Azure BLOB storage. Simply create an "Aspera Transfer Service" instance (https://ts.asperasoft.com), which provides access to the node API. Then create a configuration for the "SHOD" instance in the configuration file: in section "shares", a configuration named: awsshod. Create another configuration for the Azure ATS instance: in section "node", named azureats. Then execute the following command:

aslmcli node download /share/sourcefile --to-folder=/destinationfolder --load-params=awsshod --transfer=node [email protected]:azureats

This will get transfer information from the SHOD instance and tell the Azure ATS instance to download files.

Create your own plugin

$ mkdir -p ~/.aspera/aslmcli/plugins
$ cat<<EOF>~/.aspera/aslmcli/plugins/test.rb
require 'asperalm/cli/plugin'
module Asperalm
  module Cli
    module Plugins
      class Test < Plugin
        def declare_options; end
        def action_list; [];end
        def execute_action; puts "Hello World!"; end
      end # Test
    end # Plugins
  end # Cli
end # Asperalm
EOF

Faspex remote sources

Faspex lacks an API to list the contents of a remote source (available in web UI). To workaround this, the node API is used, for this it is required to add a section ":storage" that links a storage name to a node config and sub path. Example:

my_faspex_conf:
  url: https://10.25.0.3/aspera/faspex
  username: admin
  password: MyPassword
  storage:
    testlaurent:
      node: my_faspex_node
      :path: /myfiles
my_faspex_node:
  url: https://10.25.0.3:9092
  username: node_faspex
  password: MyPassword

In this example, a faspex storage named "testlaurent" exists in Faspex, and is located under the docroot in "/myfiles" (this must be the same as configured in Faspex). The node configuration name is "my_faspex_node" here.

Aspera Transfer Service

First time use

Using the ATS requires an "Aspera ID" (https://id.asperasoft.com/) and have a subscription associated to it.

On first execution, the user is asked to login to Aspera ID using a web browser. This creates an "ats_id" identifier (stored in a cache file).

When only one ats_id is created, it is taken by default. Else it shall be specified with --ats-id or using a parameter set.

Note: APIs are described here: https://ts.asperasoft.com/ats-guide/getting-started-guide/# guide_transfers_create_ak

Usage

Example: create access key on softlayer:

aslmcli ats access_key create --cloud=softlayer --region=ams [email protected]:'{"storage":{"type":"softlayer_swift","container":"_container_name_","credentials":{"api_key":"value","username":"_name_:_usr_name_"},"path":"/"},"id":"_optional_id_","name":"_optional_name_"}'

Example: create access key on AWS:

aslmcli ats access_key create --cloud=aws --region=eu-west-1 [email protected]:'{"id":"testkey3","name":"laurent key AWS","storage":{"type":"aws_s3","bucket":"my-bucket","credentials":{"access_key_id":"AKIA_MY_API_KEY","secret_access_key":"my/secret/here"},"path":"/laurent"}}'

Example: create access key on Azure SAS:

aslmcli ats access_key create --cloud=azure --region=eastus [email protected]:'{"id":"testkeyazure","name":"laurent key azure","storage":{"type":"azure_sas","credentials":{"shared_access_signature":"https://xxxx.blob.core.windows.net/..."},"path":"/"}}'

Example: create access key on Azure:

aslmcli ats access_key create --cloud=azure --region=eastus [email protected]:'{"id":"testkeyazure","name":"laurent key azure","storage":{"type":"azure","credentials":{"account":"myaccount","key":"myaccesskey","storage_endpoint":"myblob"},"path":"/"}}'

delete all my access keys:

for k in $(aslmcli ats access_key list --field=id --format=csv);do aslmcli ats access_key id $k delete;done

Protection against parallel execution

If it is required to run a command as a single instance, it is possible to protect with parameter: --lock-port=nnnn.

This opens a local TCP server port, and fails if this port is already used, providing a local lock.

Preview generation for Aspera Files

The tool requires the following external tools:

  • ImageMagick : convert composite
  • OptiPNG : optipng
  • FFmpeg : ffmpeg ffprobe
  • Libreoffice : libreoffice

Preview Command

The preview plugin allows generation of preview files for Aspera Files for on-premise nodes. (thumbnails and video previews)

The preview generator creates/updates a preview for files located on an access key main "storage root". Several candidate detection methods are supported.

This version requires ES 3.7.4+

This is related to:

https://aspera.asperafiles.com/helpcenter/admin/organization/installing-files-preview-maker

Configuration

Like any aslmcli commands, parameters can be passed on command line or using a configuration parameter set. Example using a parameter set:

$ aslmcli config id default set preview my_files_access_key
$ aslmcli config id my_files_access_key update --url=https://localhost:9092 --username=my_access_key --password=my_secret

Once can check if the access key is well configured using:

$ aslmcli -Pmy_files_access_key node browse /

Execution

The tool intentionally supports only a "finite execution time" mode, i.e. it needs to be run regularly to create or update preview files. I.e. it does not implement a loop. Running preview generation on a regular basis shall be done using the operating system scheduler (e.g. Cron on Linux, or Task Scheduler on Windows). To prevent parallel execution of the task, one can use either the protection offered by the scheduler, if there is one, or use the parameter: --lock-port=12345.

Candidate detection

The tool will find candidates for preview generation using three commands:

  • events : only recently uploaded files will be tested
  • scan : deeply scan all files under the access key's "storage root"
  • folder : same as scan, but only on the specified folder's "file identifier"

Creation/Update

Once candidate are selected, once candidates are selected, a preview is always generated if it does not exist already, else if a preview already exist, it will be generated using one of three overwrite method:

  • always : preview is always generated, even if it already exists and is newer than original
  • never : preview is generated only if it does not exist already
  • mtime : preview is generated only if the original file is newer than the existing

Access to original files and preview creation

Standard open source tools are used to create thumnails and video previews. Those tools require that original files are accessible in the local file system and also write generated files on the local file system. The tool provides 2 ways to read and write files with the parameter: --file-access

If the preview generator is run on a system that has direct access to the file system, then the value local can be used.

If the preview generator does not have access to files on the file system (it is remote, no mount, or is an object storage), then the original file is first downloaded, then the result is uploaded, use method remote.

Options

To skip some folders, use the option : --skip-folders, note that it expects a list of path starting with slash, use the @json: notation, example:

$ aslmcli preview scan [email protected]:'["/not_here"]'

Examples of use

on command line:

aslmcli preview event --skip-types=office --file-access=remote --overwrite=always --iteration-file=/tmp/restart.txt --lock-port=12345

with crontab:

2-59 * * * * su -s /bin/bash - xfer -c 'timeout 10m aslmcli preview event --skip-types=office --lock-port=12345 --log-level=info --logger=syslog --iteration-file=/tmp/preview_restart.txt'
0 * * * *    su -s /bin/bash - xfer -c 'timeout 30m aslmcli preview scan  --skip-types=office --lock-port=12345 --log-level=info --logger=syslog'

External tools: Linux

Here shown on Redhat/CentOS

  • Imagemagick and optipng:
yum install -y ImageMagick optipng
  • FFmpeg
pushd /tmp
wget http://johnvansickle.com/ffmpeg/releases/ffmpeg-release-64bit-static.tar.xz
mkdir -p /opt/
cd /opt/
tar xJvf /tmp/ffmpeg-release-64bit-static.tar.xz
ln -s ffmpeg-* ffmpeg
ln -s /opt/ffmpeg/{ffmpeg,ffprobe} /usr/bin
popd
  • Libreoffice
yum install libreoffice
  • Xvfb

Although libreoffice is run headless, older versions may require an X server. If you get error running libreoffice headless, then install Xvfb:

yum install libreoffice Xvfb
cat<<EOF>/etc/init.d/xvfb 
# !/bin/bash
# chkconfig: 345 95 50
# description: Starts xvfb on display 42 for headless Libreoffice
if [ -z "\$1" ]; then
  echo "\`basename \$0\` {start|stop}"
  exit
fi
case "\$1" in
start) /usr/bin/Xvfb :42 -screen 0 1280x1024x8 -extension RANDR&;;
stop) killall Xvfb;;
esac
EOF
chkconfig xvfb on
service xvfb start

Sample commands

Some commands used in unit testing:

aslmcli
aslmcli --no-default node --url=my_url_here --username=my_username_here --password=my_password_here delete /500M.dat
aslmcli --no-default node --url=my_url_here --username=my_username_here --password=my_password_here upload --to-folder=/ 500M.dat [email protected]:'{"precalculate_job_size":true}' --transfer=node [email protected]:'{"url":"my_url_here","username":"my_username_here","password":"my_password_here"}' 
aslmcli --version)
aslmcli ats access_key create --cloud=aws --region=eu-west-1 [email protected]:'{"id":"testkey3","name":"laurent key AWS","storage":{"type":"aws_s3","bucket":"sedemo-ireland","credentials":{"access_key_id":"AKIAIDSWKOSIM7XUVCJA","secret_access_key":"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"},"path":"/laurent"}}'
aslmcli ats access_key create --cloud=softlayer --region=ams [email protected]:'{"id":"testkey2","name":"laurent key","storage":{"type":"softlayer_swift","container":"laurent","credentials":{"api_key":"my_api_key_here","username":"my_username_here"},"path":"/"}}'
aslmcli ats access_key id testkey2 delete
aslmcli ats access_key id testkey2 node browse /
aslmcli ats access_key id testkey2 server
aslmcli ats access_key id testkey3 delete
aslmcli ats access_key list --fields=name,id,secret
aslmcli ats api_key list
aslmcli ats api_key repository list
aslmcli ats server id 1f412ae7-869a-445c-9c05-02ad16813be2
aslmcli ats server list clouds
aslmcli ats server list instance --cloud=aws --region=eu-west-1 
aslmcli ats server list provisioned
aslmcli ats subscriptions
aslmcli client connect id 'Aspera Connect for Windows' info
aslmcli client connect id 'Aspera Connect for Windows' links id 'Windows Installer' download --to-folder=.
aslmcli client connect id 'Aspera Connect for Windows' links list
aslmcli client connect list
aslmcli client location
aslmcli config id conf_name delete
aslmcli config id conf_name initialize @json:'{"p1":"v1","p2":"v2"}'
aslmcli config id conf_name set param value
aslmcli config id conf_name show
aslmcli config id conf_name update --p1=v1 --p2=v2
aslmcli config id default set shares conf_name
aslmcli config list
aslmcli config overview
aslmcli console transfer current list  --insecure=yes
aslmcli erb README.erb.md > README.md
aslmcli faspex package list --fields=delivery_id --format=csv --box=sent|tail -n 1) --box=sent
aslmcli faspex package list --insecure=yes
aslmcli faspex package send --load-params=reset --url=my_url_here --username=my_username_here --password=my_password_here --insecure=yes --note="my note" --title="my title" --recipient="[email protected]" ~/200KB.1
aslmcli faspex package send sample_file.bin --insecure=yes --note="my note" --title="my title" --recipient="[email protected]"
aslmcli faspex recv_publink 'https://faspex.mycompany.com/aspera/faspex/external_deliveries/78780?passcode=a003aaf2f53e3869126b908525084db6bebc7031' --insecure=yes
aslmcli files admin events
aslmcli files admin resource node id 5560 do browse / --secret=my_secret_here
aslmcli files admin resource workspace list
aslmcli files admin set_client_key ERuzXGuPA @file:$(APIKEY)
aslmcli files package list
aslmcli files package recv VleoMSrlA
aslmcli files package send sample_file.bin --note="my note" --title="my title" --recipient="[email protected]"
aslmcli files repo browse /
aslmcli files repo download /200KB.1 --to-folder=sample_dest_folder --download=node
aslmcli files repo download /200KB.1 --to-folder=sample_dest_folder --transfer=connect
aslmcli files repo upload sample_file.bin --to-folder=/
aslmcli node async id 1 counters 
aslmcli node async id 1 summary 
aslmcli node async list
aslmcli node browse / --insecure=yes
aslmcli node delete sample_dest_folder200KB.1 --insecure=yes
aslmcli node download sample_dest_folder200KB.1 --to-folder=sample_dest_folder --insecure=yes
aslmcli node service create @json:'{"id":"service1","type":"WATCHD","run_as":{"user":"user1"}}'
aslmcli node service id service1 delete
aslmcli node service list
aslmcli node upload sample_file.bin --to-folder=sample_dest_folder --insecure=yes
aslmcli orchestrator info
aslmcli orchestrator plugins
aslmcli orchestrator processes
aslmcli orchestrator workflow id 10 inputs
aslmcli orchestrator workflow id 10 start [email protected]:'{"Param":"laurent"}'
aslmcli orchestrator workflow id 10 start [email protected]:'{"Param":"laurent"}' --result=ResultStep:Complete_status_message
aslmcli orchestrator workflow id 10 status
aslmcli orchestrator workflow list
aslmcli orchestrator workflow status
aslmcli preview events --skip-types=office
aslmcli preview scan --skip-types=office --log-level=info
aslmcli preview test ~/Documents/Samples/anatomic-2k/TG18-CH/TG18-CH-2k-01.dcm --log-level=debug png --video=clips
aslmcli server browse /
aslmcli server cp /Upload/200KB.1 /Upload/200KB.2
aslmcli server delete /Upload/200KB.1
aslmcli server delete /Upload/to.delete
aslmcli server df
aslmcli server download /Upload/200KB.1 --to-folder=sample_dest_folder
aslmcli server du /
aslmcli server info
aslmcli server md5sum /Upload/200KB.1
aslmcli server mkdir /Upload/123
aslmcli server mv /Upload/200KB.2 /Upload/to.delete
aslmcli server rm /Upload/123
aslmcli server upload sample_file.bin --to-folder=/Upload
aslmcli shares repository browse / --insecure=yes
aslmcli shares repository delete /n8-sh1/200KB.1 --insecure=yes
aslmcli shares repository download /n8-sh1/200KB.1 --to-folder=sample_dest_folder --insecure=yes
aslmcli shares repository upload sample_file.bin --to-folder=/n8-sh1 --insecure=yes
aslmcli shares2 appinfo
aslmcli shares2 organization list
aslmcli shares2 project list --organization=Sport
aslmcli shares2 repository browse /
aslmcli shares2 userinfo

...and more

Usage

$ aslmcli -h
NAME
    aslmcli -- a command line tool for Aspera Applications (v0.6.5)

        --interactive=ENUM           use interactive input of missing params. Values=(yes,no), current=yes
        --ask-options=ENUM           ask even optional options. Values=(yes,no), current=no
SYNOPSIS
    aslmcli COMMANDS [OPTIONS] [ARGS]

DESCRIPTION
    Use Aspera application to perform operations on command line.
    OAuth 2.0 is used for authentication in Files, Several authentication methods are provided.
    Documentation and examples: https://rubygems.org/gems/asperalm
    execute: aslmcli conf doc

COMMANDS
    First level commands: config
    Note that commands can be written shortened (provided it is unique).

OPTIONS
    Options begin with a '-' (minus), and value is provided on command line.
    Special values are supported beginning with special prefix, like: @val: @file: @env: @["base64", "json", "zlib"]:.
    Dates format is 'DD-MM-YY HH:MM:SS', or 'now' or '-<num>h'

ARGS
    Some commands require mandatory arguments, e.g. a path.

OPTIONS: global
    -h, --help                       Show this message.
        --show-config                Display parameters used for the provided action.
    -g, --gui-modeTYPE               method to start browser. Values=(text,graphical), current=graphical
        --insecure=ENUM              do not validate HTTPS certificate. Values=(yes,no), current=no
        --flat-hash=ENUM             display hash values as additional keys. Values=(yes,no), current=yes
        --log-level=ENUM             Log level. Values=(warn,debug,info,error,fatal,unknown), current=warn
        --logger=ENUM                log method. Values=(stderr,stdout,syslog), current=
        --format=ENUM                output format. Values=(table,ruby,json,jsonpp,yaml,csv), current=table
        --transfer=ENUM              type of transfer. Values=(ascp,connect,node), current=ascp
        --config-file=VALUE          read parameters from file in YAML format, current=/Users/laurent/.aspera/aslmcli/config.yaml
    -P, --load-paramsVALUE           load the named configuration from current config file, use "none" to avoid loading the default configuration
        --fasp-folder=VALUE          specify where to find FASP (main folder), current=
        --transfer-node=VALUE        name of configuration used to transfer when using --transfer=node
        --fields=VALUE               comma separated list of fields, or ALL, or DEF
        --fasp-proxy=VALUE           URL of FASP proxy (dnat / dnats)
        --http-proxy=VALUE           URL of HTTP proxy (for http fallback)
    -r, --rest-debug                 more debug for HTTP calls
    -N, --no-default                 do not load default configuration
    -v, --version                    display version
        --ts=VALUE                   override transfer spec values (hash, use @json: prefix), current={}
        --to-folder=VALUE            destination folder for downloaded files, current=
        --lock-port=VALUE            prevent dual execution of a command, e.g. in cron
        --use-product=VALUE          which local product to use for ascp

        --interactive=ENUM           use interactive input of missing params. Values=(yes,no), current=yes
        --ask-options=ENUM           ask even optional options. Values=(yes,no), current=no
COMMAND: shares
SUBCOMMANDS: repository, admin
OPTIONS:
        --url=VALUE                  URL of application, e.g. https://org.asperafiles.com
        --username=VALUE             username to log in
        --password=VALUE             user's password

        --interactive=ENUM           use interactive input of missing params. Values=(yes,no), current=yes
        --ask-options=ENUM           ask even optional options. Values=(yes,no), current=no
COMMAND: node
SUBCOMMANDS: events, space, info, mkdir, mklink, mkfile, rename, delete, browse, upload, download, postprocess, stream, transfer, cleanup, forward, access_key, watch_folder, service, async, central
OPTIONS:
        --url=VALUE                  URL of application, e.g. https://org.asperafiles.com
        --username=VALUE             username to log in
        --password=VALUE             user's password
        --persistency=VALUE          persistency file (cleanup,forward)
        --filter-transfer=VALUE      Ruby expression for filter at transfer level (cleanup)
        --filter-file=VALUE          Ruby expression for filter at file level (cleanup)
        --transfer-filter=VALUE      JSON expression for filter on API request
        --parameters=VALUE           creation parameters (hash, use @json: prefix), current=

        --interactive=ENUM           use interactive input of missing params. Values=(yes,no), current=yes
        --ask-options=ENUM           ask even optional options. Values=(yes,no), current=no
COMMAND: orchestrator
SUBCOMMANDS: info, workflow, plugins, processes
OPTIONS:
        --url=VALUE                  URL of application, e.g. https://org.asperafiles.com
        --username=VALUE             username to log in
        --password=VALUE             user's password
        --params=VALUE               parameters hash table, use @json:{"param":"value"}
        --result=VALUE               specify result value as: 'work step:parameter'
        --synchronous=ENUM           work step:parameter expected as result. Values=(yes,no), current=no

        --interactive=ENUM           use interactive input of missing params. Values=(yes,no), current=yes
        --ask-options=ENUM           ask even optional options. Values=(yes,no), current=no
COMMAND: ats
SUBCOMMANDS: server, api_key, subscriptions, access_key
OPTIONS:
        --ats-id=VALUE               ATS key identifier (ats_xxx)
        --params=VALUE               Parameters hash for access key (@json:)
        --cloud=VALUE                Cloud provider
        --region=VALUE               Cloud region

        --interactive=ENUM           use interactive input of missing params. Values=(yes,no), current=yes
        --ask-options=ENUM           ask even optional options. Values=(yes,no), current=no
COMMAND: client
SUBCOMMANDS: installation, monitor, location, connect
OPTIONS:

        --interactive=ENUM           use interactive input of missing params. Values=(yes,no), current=yes
        --ask-options=ENUM           ask even optional options. Values=(yes,no), current=no
COMMAND: faspex
SUBCOMMANDS: package, dropbox, recv_publink, source, me
OPTIONS:
        --url=VALUE                  URL of application, e.g. https://org.asperafiles.com
        --username=VALUE             username to log in
        --password=VALUE             user's password
        --recipient=VALUE            package recipient
        --title=VALUE                package title
        --note=VALUE                 package note
        --metadata=VALUE             package metadata hash value (use @json:)
        --source-name=VALUE          create package from remote source (by name)
        --box=ENUM                   package box. Values=(inbox,sent,archive), current=inbox

        --interactive=ENUM           use interactive input of missing params. Values=(yes,no), current=yes
        --ask-options=ENUM           ask even optional options. Values=(yes,no), current=no
COMMAND: shares2
SUBCOMMANDS: repository, organization, project, team, share, appinfo, userinfo
OPTIONS:
        --organization=VALUE         organization
        --project=VALUE              project
        --share=VALUE                share

        --interactive=ENUM           use interactive input of missing params. Values=(yes,no), current=yes
        --ask-options=ENUM           ask even optional options. Values=(yes,no), current=no
COMMAND: preview
SUBCOMMANDS: scan, events, folder, check, test
OPTIONS:
        --url=VALUE                  URL of application, e.g. https://org.asperafiles.com
        --username=VALUE             username to log in
        --password=VALUE             user's password
        --file-access=ENUM           how to read and write files in repository. Values=(local,remote), current=local
        --skip-types=VALUE           skip types in comma separated list
        --overwrite=ENUM             when to generate preview file. Values=(always,never,mtime), current=mtime
        --previews-folder=VALUE      preview folder in files
        --iteration-file=VALUE       path to iteration memory file
        --folder-reset-cache=ENUM    reset folder cache. Values=(no,header,read), current=no
        --temp-folder=VALUE          path to temp folder
        --skip-folders=VALUE         list of folder to skip
        --video=ENUM                 method to generate video. Values=(reencode,clips,preview), current=reencode
        --vid-offset-seconds=VALUE   generation parameter
        --vid-size=VALUE             generation parameter
        --vid-framecount=VALUE       generation parameter
        --vid-blendframes=VALUE      generation parameter
        --vid-framepause=VALUE       generation parameter
        --vid-fps=VALUE              generation parameter
        --vid-mp4-size-reencode=VALUE
                                     generation parameter
        --clips-offset-seconds=VALUE generation parameter
        --clips-size=VALUE           generation parameter
        --clips-length=VALUE         generation parameter
        --clips-count=VALUE          generation parameter
        --thumb-mp4-size=VALUE       generation parameter
        --thumb-img-size=VALUE       generation parameter
        --thumb-offset-fraction=VALUE
                                     generation parameter
        --validate-mime=ENUM         use magic number validation. Values=(no,yes), current=no
        --check-extension=ENUM       check extra file extensions. Values=(no,yes), current=yes

        --interactive=ENUM           use interactive input of missing params. Values=(yes,no), current=yes
        --ask-options=ENUM           ask even optional options. Values=(yes,no), current=no
COMMAND: server
SUBCOMMANDS: nodeadmin, userdata, configurator, download, upload, browse, delete, rename, ls, rm, mv, du, info, mkdir, cp, df, md5sum
OPTIONS:
        --url=VALUE                  URL of application, e.g. https://org.asperafiles.com
        --username=VALUE             username to log in
        --password=VALUE             user's password
        --ssh-keys=VALUE             PATH_ARRAY is @json:'["path1","path2"]'

        --interactive=ENUM           use interactive input of missing params. Values=(yes,no), current=yes
        --ask-options=ENUM           ask even optional options. Values=(yes,no), current=no
COMMAND: files
SUBCOMMANDS: package, repository, faspexgw, admin
OPTIONS:
        --download-mode=ENUM         download mode. Values=(fasp,node_http), current=fasp
    -t, --authTYPE                   type of authentication. Values=(basic,web,jwt,url_token), current=jwt
        --url=VALUE                  URL of application, e.g. http://org.asperafiles.com
        --username=VALUE             username to log in
        --password=VALUE             user's password
        --private-key=VALUE          RSA private key PEM value for JWT (prefix file path with @val:@file:)
        --workspace=VALUE            name of workspace
        --recipient=VALUE            package recipient
        --title=VALUE                package title
        --note=VALUE                 package note
        --secret=VALUE               access key secret for node
        --query=VALUE                for json query

        --interactive=ENUM           use interactive input of missing params. Values=(yes,no), current=yes
        --ask-options=ENUM           ask even optional options. Values=(yes,no), current=no
COMMAND: console
SUBCOMMANDS: transfer
OPTIONS:
        --url=VALUE                  URL of application, e.g. https://org.asperafiles.com
        --username=VALUE             username to log in
        --password=VALUE             user's password
        --filter-from=DATE           only after date
        --filter-to=DATE             only before date

Documentation : http://www.rubydoc.info/gems/asperalm


Note that actions and parameter values can be written in short form.

asession

This gem comes with a second executable tool providing a simplified standardized interface to start a FASP session: asession.

It aims at simplifying the startup of a FASP session from a programmatic stand point as formating a transfer spec is:

  • common to Aspera Node API (HTTP POST /ops/transfer)
  • common to Aspera Connect API (browser javascript startTransfer)
  • easy to generate by using any third party language specific JSON library

This makes it easy to integrate with any language provided that one can spawn a sub process, write to its STDIN, read from STDOUT, generate and parse JSON.

The tool expect one single argument: a transfer spec (see Transfer spec) hash value using the extended argument syntax ( see Option and Argument values).

If not argument is provided, it assumes a value of: @json:@stdin, i.e. a JSON formated transfer spec on stdin.

Note that if JSON is the format, one has to specify @json: to tell the tool to decode the hash using JSON.

During execution, it generates all low level events, one per line, in JSON format on stdout.

Comparison:

feature/toolasessionascpFaspManager
language integrationanyanyC/C++
C#/.net
Go
Python
java
componentsascp binary
+Ruby
ascp binaryascp binary
+library
startupJSON on stdin
(standard APIs:
JSON.generate
Process.spawn)
complex command line argumentsAPI
eventsJSON on stdoutnone by default
or need to open management port
and proprietary text syntax
callback

Examples of use:

  • Shell
MY_TSPEC='{"remote_host":"demo.asperasoft.com","remote_user":"asperaweb","ssh_port":33001,"remote_password":"demoaspera","direction":"receive","destination_root":"./test.dir","paths":[{"source":"/aspera-test-dir-tiny/200KB.1"}]}'

echo "${MY_TSPEC}"|asession
$ asession -h
USAGE
    asession <session information>
    <session information> is a JSON formatted transfer spec, starting with "@json:".
    The value can be either:
    o the JSON description itself
    o @stdin, if the JSON is provided from stdin
    o @file:<path>, if the JSON is provided from a file
Note: debug information can be placed on STDERR, using the "EX_loglevel" parameter in transfer spec (debug=0)
EXAMPLES
    asession @json:'{"remote_host":"demo.asperasoft.com","remote_user":"asperaweb","ssh_port":33001,"remote_password":"demoaspera","direction":"receive","destination_root":"./test.dir","paths":[{"source":"/aspera-test-dir-tiny/200KB.1"}]}'
    echo '{"remote_host":...}'|asession @json:@stdin


BUGS

This is best effort code without official support, dont expect full capabilities. This code is not supported by IBM/Aspera. You can contact the author for bugs or features.

If you get message: "OpenSSH keys only supported if ED25519 is available" this means that you do not have ruby support for ED25519 SSH keys. You may either install the suggested Gems, or remove your ed25519 key from your .ssh folder to solve the issue.

TODO

Contributing

Create your own plugin !

Send comments !