FTP file output plugin for Embulk

Build Status

This plugin support FTP, FTPES(FTPS explicit), FTPS(FTPS implicit) and doesn't support SFTP.

If you want to use SFTP, please use embulk-output-sftp.


  • Plugin type: file input
  • Resume supported: no
  • Cleanup supported: yes


  • host: FTP server address (string, required)
  • port: FTP server port number (integer, default: 21. 990 if ssl is true and ssl_explicit is false)
  • user: user name to login (string, optional)
  • password: password to login (string, default: "")
  • path_prefix prefix of target files (string, required)
  • sequence_format Format for sequence part of output files (string, default: ".%03d.%02d")
  • file_ext e.g. "csv.gz, json.gz" (string, required)
  • passive_mode: use passive mode (boolean, default: true)
  • ascii_mode: use ASCII mode instead of binary mode (boolean, default: false)
  • ssl: use FTPS (SSL encryption). (boolean, default: false)
  • ssl_explicit use FTPS(explicit) instead of FTPS(implicit). (boolean, default:true)
  • ssl_verify: verify the certification provided by the server. By default, connection fails if the server certification is not signed by one the CAs in JVM's default trusted CA list. (boolean, default: true)
  • ssl_verify_hostname: verify server's hostname matches with provided certificate. (boolean, default: true)
  • ssl_trusted_ca_cert_file: if the server certification is not signed by a certificate authority, set path to the X.508 certification file (pem file) of a private CA (string, optional)
  • ssl_trusted_ca_cert_data: similar to ssl_trusted_ca_cert_file but embed the contents of the PEM file as a string value instead of path to a local file (string, optional)

FTP / FTPS default port number

FTP and FTPS server listens following port number(TCP) as default.

Please be sure to configure firewall rules.

FTP FTPS(explicit) = FTPES FTPS(implicit) = FTPS
Control channel port 21 21 990 (*1)
Data channel port (*2) 20 20 989
  1. If you're using both of FTPS(implicit) and FTP, server also use 21/TCP for FTP.
  2. If you're using passive mode, data channel port can be taken between 1024 and 65535.


Simple FTP:

  type: ftp
  host: ftp.example.net
  port: 21
  user: anonymous
  path_prefix: /ftp/file/path/prefix
  file_ext: csv
    type: csv.gz
    header_line: false
  - {type: gzip}

FTPS encryption without server certificate verification:

  type: ftp
  host: ftp.example.net
  port: 21
  user: anonymous
  password: "mypassword"

  ssl: true
  ssl_verify: false

  path_prefix: /ftp/file/path/prefix
  file_ext: csv

FTPS encryption with server certificate verification:

  type: ftp
  host: ftp.example.net
  port: 21
  user: anonymous
  password: "mypassword"

  ssl: true
  ssl_verify: true

  ssl_verify_hostname: false   # to disable server hostname verification (optional)

  # if the server use self-signed certificate, or set path to the pem file (optional)
  ssl_trusted_ca_cert_file: /path/to/ca_cert.pem

  # or embed contents of the pem file here (optional)
  ssl_trusted_ca_cert_data: |
      -----BEGIN CERTIFICATE-----
      -----END CERTIFICATE-----

  path_prefix: /ftp/file/path/prefix
  file_ext: csv


$ ./gradlew gem  # -t to watch change of files and rebuild continuously


Firstly install Docker and Docker compose then docker-compose up -d, so that an FTP server will be locally launched then you can run tests with ./gradlew test.

$ docker-compose up -d
Creating network "embulk-output-ftp_default" with the default driver
Creating embulk-output-ftp_server  ... done
Creating embulk-output-ftps_server ... done

$ docker-compose ps
         Name              Command     State                       Ports
embulk-output-ftp_server    /usr/sbin/run-vsftpd.sh          Up      20/tcp,>21/tcp,>65000/tcp,>65001/tcp,>65002/tcp,>65003/tcp,                        
embulk-output-ftps_server   /usr/sbin/run-vsftpd-with- ...   Up      20/tcp,>21/tcp,>65005/tcp,>65006/tcp,>65007/tcp,>65008/tcp    

$ ./gradlew test  # -t to watch change of files and rebuild continuously

If you want to use other FTP server to test, configure the following environment variables.

FTP_TEST_HOST (default: localhost)
FTP_TEST_PORT (default: 11021)
FTP_TEST_SSL_PORT (default:990)
FTP_TEST_USER (default: scott)
FTP_TEST_PASSWORD (default: tigger)

If you're using Mac OS X El Capitan and GUI Applications(IDE), like as follows.

$ vi ~/Library/LaunchAgents/environment.plist
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
      launchctl setenv FTP_TEST_HOST ftp.example.com
      launchctl setenv FTP_TEST_USER username
      launchctl setenv FTP_TEST_PASSWORD password
      launchctl setenv FTP_TEST_SSL_TRUSTED_CA_CERT_FILE /path/to/cert.pem
      -----END CERTIFICATE-----"

$ launchctl load ~/Library/LaunchAgents/environment.plist
$ launchctl getenv FTP_TEST_HOST //try to get value.

Then start your applications.


This program is forked from embulk-input-ftp and originally written by @frsyuki, modified by @sakama.