jscp, jtool scp

A platform agent tool used to copy files using the SCP or SFTP protocol. You can call it as jscp or jtool scp. When used from within a process, it can retrieve private key passphrases and user passwords from the Redwood login and passphrase credential objects.

It adheres to the following RFCs:

• rfc4251 - The Secure Shell (SSH) Protocol Architecture
• [rfc4252]https://www.rfc-editor.org/rfc/rfc4252) - The Secure Shell (SSH) Authentication Protocol
• rfc4253 - The Secure Shell (SSH) Transport Layer Protocol
• rfc4254 - The Secure Shell (SSH) Connection Protocol

Syntax

jscp  [-h|-?|-help] [-l <loglevel>] [-f <logfile>] <command>

  Option           Default                Purpose
  ---------------- ---------------------- -----------------------------------
  -h|-?|-help                             Show this help and exit
  -l <loglevel>    ${JCS_LOGLEVEL:-info}  Set the logging level; defaults to environment variable ${JCS_LOGLEVEL}
                                          or info when the environment variable is not set.
  -f <logfile>     ${JCS_LOGFILE:-stderr} Log to file instead of stderr; defaults to environment variable
                                          ${JCS_LOGFILE} or stderr when the environment variable is not set.
  <[-]command>...                         Command(s) passed to jscp

where command represents one of the following:

[options] [user@]host:source target
[options] source [source...] [user@]host:target
[options] -ls [user@]host:filespec

Options:
  -V              print version information and exit
  -pgpfp          print PGP key fingerprints and exit
  -p              preserve file attributes
  -q              quiet, don't show statistics
  -r              copy directories recursively
  -v              show verbose messages
  -load sessname  load settings from saved session
  -P port         connect to specified port
  -u user         connect with specified username
  -pw passw       login with specified password
  -1 -2           force use of particular SSH protocol version
  -4 -6           force use of IPv4 or IPv6
  -C              enable compression
  -i key          private key file for authentication
  -[no]agent      en/disable use of authentication agent
  -[no]batch      en/disable all interactive prompts
  -unsafe         allow server-side wildcards (DANGEROUS)
  -storekey       store a new or updated server key in cache
  -sftp           force use of SFTP protocol
  -scp            force use of SCP protocol

SSH Host Keys

jscp no longer accesses the ${HOME}/.ssh/known_hosts file. jtool scp reads the server authentication host key from ${HOME}/.jtool/sshhostkeys (UNIX) or HKEY_CURRENT_USER\Software\Redwood\Jtool\SshHostKeys (Windows).

You run jtool scp interactively with the -nobatch argument in order to store the remote host key in ${HOME}/.jtool/sshhostkeys (UNIX) or HKEY_CURRENT_USER\Software\Redwood\Jtool\SshHostKeys (Windows); this has to be done for every remote host to which jtool scp connects.

When running in a process, SSH_ASKPASS will default to jgetcredential. Therefore, during user authentication, ssh will attempt to retrieve any required private key passphrase or user password from the Redwood Server.

jtool scp host keys

Jtool SSH host keys are stored in the following format:

In ${HOME}/.jtool/sshhostkeys:

<keytype>@<port>:<host> <key>

In HKEY_CURRENT_USER\Software\Redwood\Jtool\SshHostKeys:

Name	Type	Data
<keytype>@<port>:<host>	REG_SZ	<key>

• <keytype - the public key algorithm
• <port> - port the OpenSSH-compatible server is listening on
• <host> - hostname or IP address in square brackets, such as pr1.example.com
• <key> - public key in PuTTY format

You use PuTTYGen to convert between OpenSSH and PuTTY public key formats.

OpenSSH host keys

The format of ${HOME}/.ssh/known_hosts in OpenSSH_7.4p1 uses the following syntax (provided HashKnownHosts is set to no ):

<host>:<port>[,<host>:<port>]* <keytype> <key>

where

• <host> - hostname or IP address in square brackets, such as [pr1.example.com]
• <port> - port the OpenSSH-compatible server is listening on
• <keytype} - the public key algorithm
• <key> - public key in OpenSSH format

Example:

[pr1.example.com]:2222,[10.0.4.59]:2222,[10.0.4.57]:2222 ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKjLzSTr4DafbLQQPO+S+dGjLOoQue7z8BCMfICda012PkpGcS+97YaCE7rg0RzoFW4SPz0wpN6gFM05g6Fsomo=

Using PuTTYGen you can convert these keys to PuTTY syntax and simply copy them into the sshhostkeys file or create a String Value in the Windows registry as outlined above.

Configuring Certificate Authentication

For increased security, you can opt for key based authentication coupled with a passphrase stored in a Redwood Server passphrase credential. In order to verify that the correct credential was used, you may specify the '-l debug' switch with jscp, or JCS_LOGLEVEL=debug with ssh in order to enable debugging.

Fall back to Username/Password Authentication

Fall back to username/password authentication using a credential of type login, if the SSH server refuses to authenticate the client using a private key and passphrase (invalid/unknown key, incorrect passphrase...) :

jtool scp -i <private keyfile> <otherargs> || jtool scp -u <username> <otherargs>

The jtool scp binary uses the putty implementation for keys and certificates, so you will have to install putty and puttygen on a system to generate the keys and certificates. Note that these tools are not needed on the systems where you run jtool scp; the keyfiles are required.

Note that these instructions are provided for your convenience and use a simple key with 4096 bits. Numerous resources are available online with more in-depth information on how to create more secure keys.

Generate a Putty Keyfile

To generate a key using puttygen, note that after you have issued the command, you will have to move your mouse for a minute or two, until you see +++ being printed. The passphrase you enter should ideally be a sentence, with multiple words; you store the passphrase in a credential in Redwood Server, the Username of the credential should be set to the value of the comment field ( -C ).

$ puttygen -t rsa -b 4096 -C "jdoe" -o jdoe.ppk
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+++++
Enter passphrase to save key:
Re-enter passphrase to verify:

Resulting public/private key pair:

$ cat example.ppk
PuTTY-User-Key-File-2: ssh-rsa
Encryption: aes256-cbc
Comment: example
Public-Lines: 12
AAAAB3NzaC1yc2EAAAABJQAAAgEAg5ESdWYAARvMTdXkZSEsdjcB1H6fxsqUipwY
rJTaHKjGU0VvB/LdIWlSVx65vIMEtPatbK99QhczuAWoX95gIDmIRoyuCXVxp6Vi
zpVRpLv9+ni3GRm39RKiPvWLXZ+8opAU73cpwFONoJQ+3oW7n/TuGfibLJEXUch5
W3zZ4lyJQ+xMIPUcdVswOxD3a3O6HuSU94z7AwHEhOE4ABHMrlBE1t6wx00cs0E5
eKfvGxt+ExjJp6X+Jv81nFFCrj5A+V508JqicoLLRLLCkxfFGiFEjVLinxh46+HJ
8nwE3PRjTvt0sCS/q2Sz3sHF9pYC89/k+BD5YU0KVnN9hZZZpaH0JfH0AIDzOMtT
O5ymn/zE38JQEM0LbuJ8YM7bg/O1wrlnUtqkHqn29OrB6y75VgPaw26cEbDSDpo/
W1d3/HFIWifp8z0SJ7jPp0Sidle/7B+VWc/Vv3ZbBhDSp6/0LUDH8CtV1aJaLhUL
1U2D3yufZLMIUvwABOOipKI4Zmpdu5Y5h1nZf251o9GSD3nhkh0CQdQdw1lqGcOq
p71NmAWAm6A+RD3dCoCqOlaEL/OcoJLizYPoMLUfsBwSAjQ0mNC8m9vw8HR8I0Os
2V7BNvICk3p5fiYfWbsSIBB9fkyt3VHRJRlTA7+mdzfIEfoeIy5ixqmZ6ZjD8oB6
c4sGPa8=
Private-Lines: 28
4Oe1LWcx9uvMxP+g00Yqhy+hODG9FLaTpB5zAROKsdQiNdbD6Brqwd4CPoefT4qp
Ehv5QPy4Iei/9Bt8EKCsZM7gQ5u9LFmoa6H0+IsnxcIxddETrXkBfuiiQ4g6wvar
V1o/jvSh35sJv9r1aAiMxpZEK2xN7iubhBwJ2YI57+vYPSWu3LQkQgvDpTyD0Xq/
JZKMmizz887JD+Nd58V/yfwtxla1B3kKXAwwhqYYShG5n/EpBCdoHP3ion+/FDms
J0DR1Sd8mo/IbWPd2Lhszlhcsty24uqfo2Nq8RQPJCZZZFcIsgpv2EHk5PwI3+Zh
o92m6VrBtCm/Gy03kRedT4FyYppqKZiSwvkgz9QAqP9vCa2IQGJlchjEkg06/Z13
j5NW4zHzhhACavvkRZfadsnSeAFv9D9//TAbrov2Tx20PEqiTMItvgrg8KdgT7gY
Lh/STLcSJuuI2q3AX9UzMRGdcvtQj2Cgk3KAWWWUrNzXqyTt8wg/ZmIhMtTogDVO
naIXbpukq46dyRzMJxkMHeQLJu9Wj3uQLySqXm2OUp5p1ry5k6+xvq2Cdp0lWQXn
CFOhYVpdiN9O6aV1v7b7icBit9TNqflLqZCclqNhrT/Gx1DVptMCoMCZHZ6Ft24N
mIhxbVrDKw5AlFGZpxT1/18iIcI/7cn+SLFweG9Ke1T4mxEPC9knbKi2k5YerkJ5
M2DOEXi7Ki8tpPByD59Dc7vuoybTYkVxuZCL/kAzZkdzuMJHq5MTchXdBt27p3sP
V5W9CE1+bUjhfipG32vcR6CE1Ih2mi0ZCcW8A6foBTCLjjHJNylfYXY2cXMLGEBv
oH7TgC7iIQ0Fo1G/RI9IsUYcAmluuh27pdJSH3kIx9ie2M6kXPoKRAqTrG89Ey1O
CQMFxlmmAIQ9q7JRlMPK6GG/gM9lYrBWcGwnJCpSiQvahWLOHJsDVbDZDQShsSwH
JAbmipsRnwAdcTCjNWwY1RPalHxAA5li55Mbc/5mHme6Xm4SQKsbzf2MmbD5zUFj
kdV2S/RhpDAGk3ribawh1cjA5wFmYMSnQzXVJDQrUeZEIUSbVq81RbWCNEkTwOWL
F4zMGuwqETKKIZkYHktWySST9p0C5af3sla3aVeUhkPc3Ou/bRRCqkMnBr/GCWsl
L7ezDoJpsMebydu16YQ4q2fkgpSATECpiAOajfXknjeNiR1EGcy1wLIT1QtgFdUv
CPbR3cELGv0raeCp2ba2dg8EeVi6k8cNXEV5sV+d1WPVnPIP8N3t/QTwUrj2KYIf
vTT4hcMxBUKKkBZ4v8eW8m1Uel3D9czP3RGZFZX/jm5/1oZCoNcnbfvdslz7uktx
f5HdWQHOhJgAN6DY+MpuDDnk/ybdfQkih0AXnPu9NjyJzSoNUYR9mDzts4OulpzT
6rCnsIdGCchdMM2D3zEtARgutlNCM5lITtF7jq+nR693t5/FMo+kRoP8HwzmyHTo
dS2sz4oYj4uSnLYiyfPu90jB/1HRTp9uWwOvJLzvhBI1HbhX/6FdP+JC81p4uthj
5wCBUG+DHEOmJtNMgksIS5oxF7VMcd7xn4VjMPPob3I6fIHYqxwqjd3g6mFFAEnj
Vm9wI/J57u5qflk5m0DsnvYV0tdSwb54w6TQqiE4eVP7L/bk4dZchTCZCjR07ub0
VkhpAg5kPI4vNACafAoPjo2fBuEPSiKi/Z/6MyJbhsl5gf9Jpvsp1X8MR5jEmi98
nBf25NlIFi7sdmiWOXXM7A==
Private-MAC: 35095f101ZZZ826154023d94dcZZZ4479159bbb6

You create a passphrase credential in Redwood server to store the passphrase of your certificate.

Install Putty Key in SSH Server

Installing the key into the ~/.ssh/authorized_keys file of an SSH server:

Note that you copy the public key part to the remote SSH server, in this case:

AAAAB3NzaC1yc2EAAAABJQAAAgEAg5ESdWYAARvMTdXkZSEsdjcB1H6fxsqUipwY
rJTaHKjGU0VvB/LdIWlSVx65vIMEtPatbK99QhczuAWoX95gIDmIRoyuCXVxp6Vi
zpVRpLv9+ni3GRm39RKiPvWLXZ+8opAU73cpwFONoJQ+3oW7n/TuGfibLJEXUch5
W3zZ4lyJQ+xMIPUcdVswOxD3a3O6HuSU94z7AwHEhOE4ABHMrlBE1t6wx00cs0E5
eKfvGxt+ExjJp6X+Jv81nFFCrj5A+V508JqicoLLRLLCkxfFGiFEjVLinxh46+HJ
8nwE3PRjTvt0sCS/q2Sz3sHF9pYC89/k+BD5YU0KVnN9hnfrpaH0JfH0AIDzOMtT
O5ymn/zE38JQEM0LbuJ8YM7bg/O1wrlnUtqkHqn29OrB6y75VgPaw26cEbDSDpo/
W1d3/HFIWifp8z0SJ7jPp0Sidle/7B+VWc/Vv3ZbBhDSp6/0LUDH8CtV1aJaLhUL
1U2D3yufZLMIUvwABOOipKI4Zmpdu5Y5h1nZf251o9GSD3nhkh0CQdQdw1lqGcOq
p71NmAWAm6A+RD3dCoCqOlaEL/OcoJLizYPoMLUfsBwSAjQ0mNC8m9vw8HR8I0Os
2V7BNvICk3p5fiYfWbsSIBB9fkyt3VHRJRlTA7+mdzfIEfoeIy5ixqmZ6ZjD8oB6
c4sGPa8=

You must prepend it with ssh-rsa (as specified in the keyfile) and the text must be on one line!

ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAgEAg5ESdWYAARvMTdXkZSEsdjcB1H6fxsqUipwYrJTaHKjGU0VvB/LdIWlSVx65vIMEtPatbK99QhczuAWoX[...]c4sGPa8=

Note that the keys above are for illustration purposes only and will not work.

Create Credential for Passphrase

Create a Redwood Server passphrase credential:

• Real User - jdoe (as specified in the comment of the key)
• Password - <passphrase_as_specified_during_key_generation>
• Protocol - passphrase
• Endpoint - pr1.example.com

Troubleshooting

jtool scp will log:

• secure.plib - Wrong passphrase if the server accepted the keyfile, however, the passphrase was incorrect; In this case, enable debugging and make sure the correct credential is being picked up by jtool scp.
• secure.plib - Server refused our key if the server does not know the public key; inspect the.ssh/authorized_keys file, make sure the key is on one line, starts with the correct keyword (ssh-rsa in the example above) followed by a space, followed by the key. You might have to specify the username, such as jdoe@pr1.example.com:/tmp/target_file.txt.

Advanced Settings

The jscp tool allows you to load PuTTY sessions settings using the -load parameter, jscp uses the following locations for configuration:

Windows

1. PuTTY defaults.
2. HKEY_CURRENT_USER\Software\Redwood\Jtool\Sessions\Default%20Settings - Windows default session settings.
3. HKEY_CURRENT_USER\Software\Redwood\Jtool\Sessions\<session> - Windows session settings.

UNIX

1. PuTTY defaults.
2. ${HOME}/.jtool/sessions/Default%20Settings - UNIX default settings.
3. ${HOME}/.jtool/sessions/<session> - UNIX session settings.

Session-specific settings override the Default%20Settings and PuTTY defaults.

$ jscp -load

You can use the PuTTY settings as stored in the registry (Windows) or in ${HOME}/.putty/sessions (UNIX), note, however, that a number of settings such as terminal-emulation settings do not affect jscp. The jscp tool will use the defaults for any unspecified settings.

The following are of particular interest (non-exhaustive):

KEY=ecdh,dh-gex-sha1,dh-group14-sha1,rsa,WARN,dh-group1-sha1
Ciphers=aes,chacha20,blowfish,3des,WARN,arcfour,des
SSHManualHostKeys=ecdsa-sha2-nistp256@22:devinfra=nistp256,0x8af668653d17234635708a93323e70314e74910ecab4c0c4b8897203505f0a8b,0x15bcd4d9c9d9e09e32342bdcebf508bcea566eeeeeb3954b37f2d82c3c4f3b1b

In the Windows registry, you create String values named, for example KEX, and Value is what comes after the = sign.

The following registry entry

Windows Registry Editor Version 5.00

[HKEY_CURRENT_USER\Software\Redwood\Jtool\Sessions\Custom]
"KEX"="ecdh,dh-group14-sha1,rsa,WARN,dh-group1-sha1,dh-gex-sha1"

This is equivalent to a file named ${HOME}/.jtool/sessions/Custom containing the following on UNIX:

KEY=ecdh,dh-group14-sha1,rsa,WARN,dh-group1-sha1,dh-gex-sha1

To use session settings, you specify the name of the session:

jscp -load Custom -i mykey.ppk myFile.txt jdoe@pr1.example.com:

tip

If you created a settings file/registry entry with PuTTY and wish to use that with jscp you have to replace any spaces in the session name with %20, if PuTTY has not already done this for you. You can also export the registry entry and replace SimonTatham\PuTTY with Rewdood\Jtool.

Example

Authenticate Using Login Credentials

You created a login credential with the following values:

• Real User - jdoe
• Password - ****
• Protocol - login
• Endpoint - *.example.com

You created a definition with three mandatory parameters remoteUser, remoteHost, and path, the source of the definition is as follows:

jtool scp -ls ${remoteUser}@${remoteHost}:${path}

Authentication Using Keys and Passphrases

You created the following passphrase credential:

• Real User - jdoe (as specified in the comment of the key)
• Password - <passphrase_as_specified_during_key_generation>
• Protocol - passphrase
• Endpoint - pr1.example.com

You created a definition with two mandatory parameters, remoteUser and remoteHost, as well as the following source:

jtool scp -i jdoe.ppk -storekey -ls ${remoteUser}@${remoteHost}:.

Output:

drwxr-xr--    5 example staff         170 May 25 17:47 Applications
drwx------    8 example staff         272 Aug 20 12:26 Desktop
drwx------    7 example staff         238 May 23 15:46 Documents
drwx------   38 example staff        1292 May 22 18:47 Downloads
drwx------   45 example staff        1530 May 25 17:47 Library
drwx------   10 example staff         340 May 25 17:26 Movies
drwx------    4 example staff         136 May 22 18:37 Music
drwx------    6 example staff         204 May 25 13:42 Pictures
drwxr-xr-x    6 example staff         204 Dec 30  2014 Public

Retrieving Files with the SFTP Protocol

You have a file on an SFTP server and you would like to retrieve it.

You created the following login credential:

• Real User - jdoe
• Password - ****
• Protocol - login
• Endpoint - *.example.com

You created a definition with two mandatory parameters, remoteUser and remoteHost (set to jdoe and pr1.example.com, respectively), as well as the following source:

jtool scp -sftp c:\MyFiles\myfile.txt ${remoteUser}@${remoteHost}:MyRemoteFiles/myfile

This example uses the SFTP protocol to retrieve the file.

You can specify the password as well (not recommended):

jtool scp -sftp -pw myUnsafePassword c:\MyFiles\myfile.txt ${remoteUser}@${remoteHost}:MyRemoteFiles/myfile

Troubleshooting

You are connecting to an SSH/SFTP server for the first time and the host key is not cached; the following is an example of the error you would get:

ERROR 2023-10-02 17:35:14,486 CET [73199-jscp] secure.plib - The server's host key is not cached. You have no guarantee that the server is the computer you think it is.
ERROR 2023-10-02 17:35:14,486 CET [73199-jscp] secure.plib - The server's ssh-ed25519 key fingerprint is: ssh-ed25518 256 58:4e:bd:76:1f:11:33:cd:3b:c7:a1:78:b1:6b:7a:f7
ERROR 2023-10-02 17:35:14,486 CET [73199-jscp] secure.plib - Use the -storekey argument to accept this key in batch mode
ERROR 2023-10-02 17:35:14,486 CET [73199-jscp] secure.plib - Disconnected: Aborted at host key verification
FATAL 2023-10-02 17:35:14,486 CET [73199-jscp] secure.scp - Lost connection

You need to verify the fingerprint of the host key, and then add the host key to the system's cache; one way to achieve this is running the jscp command once more with the -storekey option.

Example

You initially ran this command, from the previous example, and received the above error:

C:\ > jtool scp c:\MyFiles\myfile.txt ${remoteUser}@${remoteHost}:MyRemoteFiles/myfile

You need to add -storekey to the command once, successive commands do not require this anymore:

C:\ > jtool scp -storekey c:\MyFiles\myfile.txt ${remoteUser}@${remoteHost}:MyRemoteFiles/myfile
INFO  2023-10-02 14:43:03,880 Etc/GMT-1 [12292-jscp] secure.plib - Stored host key for pr1.example.com
INFO  2023-10-02 14:43:03,880 Etc/GMT-1 [12292-jscp] secure.plib - myfile.txt                 copied 811 bytes
INFO  2023-10-02 14:43:03,980 Etc/GMT-1 [12292-jscp] secure.plib - Server sent command exit status 0

C:\ > jtool scpi c:\MyFiles\myfile.txt ${remoteUser}@${remoteHost}:MyRemoteFiles/myfile2
INFO  2023-10-02 14:44:03,880 Etc/GMT-1 [12294-jscp] secure.plib - myfile.txt                 copied 811 bytes
INFO  2023-10-02 14:44:03,980 Etc/GMT-1 [12294-jscp] secure.plib - Server sent command exit status 0

See Also

• Command Line System Tools
• jtool ftp, jftp

scp sftp