We needed a way to automate downloads of certain files of a ftp server that is locate behind a Juniper VPN connection. See
When VPN was up and running we needed a way to download the files from the server and after the files has been downloaded we need to get them deleted on the server. We also needed to be able to exclude some files that we don’t want to download or be deleted. So wee decided to go with LFTP since we had some experience with it already.
First install lftp, since we are on Debian Buster we use apt
sudo apt install lftp
We also want all logging for lftp to be logged to a certain place to do that we need to edit ~/.lftp/rc as the user that is going to run the lftp command.
Keep in mind that the user needs write privileges in the folder where it is going to store the logfile.
set log:enabled/xfer yes set log:file/xfer /pathtolog/lftp.log
And Now we can test if it works.
Here is a simple oneliner that does what we want.
lftp -u username,password hostname -e "mirror --exclude file1 --exclude file2 --Remove-source-files --verbose remotefolderpath localfolderpath; bye"