How To Get The Latest Folder That Contains A Specific File Of Interest In Linux And Download That File Using Paramiko In Python?

I am trying to scp a specific file from a remote server to my local machine using Paramiko in Python 3. Background: There is a directory mydir on the destination machine 198.18.2.

Solution 1:

Executing scp command on the remote machine to push the file back to the local machine is an overkill. And in general relying on shell commands is very fragile approach. You better use native Python code only, to identify the latest remote file and pull it to your local machine. Your code will be way more robust and readable.

---

sftp = ssh.open_sftp()
sftp.chdir('/mydir')

files = sftp.listdir_attr()

dirs = [f for f in files if S_ISDIR(f.st_mode)]
dirs.sort(key = lambda d: d.st_mtime, reverse = True)

filename = 'email_summary.log'for d in dirs:
    print('Checking ' + d.filename)
    try:
        path = d.filename + '/' + filename
        sftp.stat(path)
        print('File exists, downloading...')
        sftp.get(path, filename)
        break
    except IOError:
        print('File does not exist, will try the next folder')

The above is based on:

• How to download only the latest file from SFTP server with Paramiko?
• Paramiko get sorted directory listing

---

Side note: Do not use AutoAddPolicy. You lose security by doing so. See Paramiko "Unknown Server".

Solution 2:

How about finding for file (not directory) using find?

find /mydir/20* -name email_summary.log | sort | tail -1

This will give you you a path to the latest file to copy.

So, your command will look like that:

scp -o StrictHostKeyChecking=no "$(find /mydir/20* -name email_summary.log | sort | tail -1)" root@198.18.1.1:/mydir/work/logs/email_summary_198.18.2.2.log