NETGEAR is aware of a growing number of phone and online scams. To learn how to stay safe click here.
Forum Discussion
atz6975
Sep 01, 2014Guide
[FR][OS 6.1.8]Ftp Backup source hidden files
Hi, I can't manage to backup from a Linux box any file that starts with "." (period), ie hidden files. Did anybody managed to backup one .something file from remote FTP to local readynas share? M...
StephenB
Sep 06, 2014Guru - Experienced User
To clarify, the OS6 FTP server on my RN102 does show hidden files to Filezilla (and any other client) in response to the normal MLSD or MLST FTP commands It does this by default. This discussion is specific to the FTP client implementation used in the backup job. Using MLST/MLSD is better than LIST for a variety of reasons, including the support for 8 bit characters in the file lists. Clients should use MLST/MLSD exclusively if the server implements them.
However, the OS6 FTP server does not tell the client that the file is hidden - and in fact it cannot, because there is no standard way to do that in FTP. The convention that ".filename" is hidden is operating system specific, and some operating systems (for instance Windows) don't use that convention.
Also, the newer commands (MLSD and MLIST) don't have options sent in the command line. There is a way to request specific "facts" in the MLST/MLSD response, that is described in RFC 3659 - including modification date, size, owner, group, and permissions. But again, not "hidden".
Finally, the FTP model is that the decision to show or not show hidden files is a server choice, not the client choice. Files are hidden for a reason, and it is up the the file repository to decide if it wants to allow them to be access/seen or not. BTW, this is true for any file or directory, not just hidden ones. The server is not obligated to show any files, or allow access to them. It can choose to filter the list any way it wishes. One reason is practical - there is no way to force server software to show all the files. There are also security aspects - files are generally hidden for a reason. This is why FileZilla makes "force showing hidden files" a server option - and it has no client option.
So
(1) there is in fact no good way to tell the FTP server to show hidden files. The old LIST command with -a might work, but might return an error or an empty list - the behavior is not defined. The new commands that support 8 bit character codes don't support that option at all.
(2) Some servers choose to show hidden files, some don't, and for some (FileZilla for instance) it is an option.
(3) if you do receive (or send) hidden files from the backup client, then when you back them up they might not stay hidden. It depends on the destination OS and file system.
My thought is that the correct behavior for the FTP backup client is to read or write all the files it finds, with the exception of "." and ".." and other known "special" file names that cannot be read or written. So if hidden files are shown in the source, the client will try to read them. And if FTP is the destination protocol, then the client will try to write them to the destination. I don't off hand see why this should be optional behavior, since the FTP client can't identify hidden files anyway. But if the FTP client is filtering out files that start with "." for some reason, then an option to defeat that filtering would be a good feature.
Adding a "LIST -a" option could create support headaches for Netgear if it is used - since there are almost certainly servers that won't accept that "-a", and will misbehave if you send it to them. And if the FTP server supports MLST/MLSD, they should be used instead, and they don't certainly don't allow "-a". So I'm thinking it isn't a good idea.
Though of course Netgear will do as it wishes - this is just what I'd do if I were them.
However, the OS6 FTP server does not tell the client that the file is hidden - and in fact it cannot, because there is no standard way to do that in FTP. The convention that ".filename" is hidden is operating system specific, and some operating systems (for instance Windows) don't use that convention.
Also, the newer commands (MLSD and MLIST) don't have options sent in the command line. There is a way to request specific "facts" in the MLST/MLSD response, that is described in RFC 3659 - including modification date, size, owner, group, and permissions. But again, not "hidden".
Finally, the FTP model is that the decision to show or not show hidden files is a server choice, not the client choice. Files are hidden for a reason, and it is up the the file repository to decide if it wants to allow them to be access/seen or not. BTW, this is true for any file or directory, not just hidden ones. The server is not obligated to show any files, or allow access to them. It can choose to filter the list any way it wishes. One reason is practical - there is no way to force server software to show all the files. There are also security aspects - files are generally hidden for a reason. This is why FileZilla makes "force showing hidden files" a server option - and it has no client option.
So
(1) there is in fact no good way to tell the FTP server to show hidden files. The old LIST command with -a might work, but might return an error or an empty list - the behavior is not defined. The new commands that support 8 bit character codes don't support that option at all.
(2) Some servers choose to show hidden files, some don't, and for some (FileZilla for instance) it is an option.
(3) if you do receive (or send) hidden files from the backup client, then when you back them up they might not stay hidden. It depends on the destination OS and file system.
My thought is that the correct behavior for the FTP backup client is to read or write all the files it finds, with the exception of "." and ".." and other known "special" file names that cannot be read or written. So if hidden files are shown in the source, the client will try to read them. And if FTP is the destination protocol, then the client will try to write them to the destination. I don't off hand see why this should be optional behavior, since the FTP client can't identify hidden files anyway. But if the FTP client is filtering out files that start with "." for some reason, then an option to defeat that filtering would be a good feature.
Adding a "LIST -a" option could create support headaches for Netgear if it is used - since there are almost certainly servers that won't accept that "-a", and will misbehave if you send it to them. And if the FTP server supports MLST/MLSD, they should be used instead, and they don't certainly don't allow "-a". So I'm thinking it isn't a good idea.
Though of course Netgear will do as it wishes - this is just what I'd do if I were them.
Related Content
NETGEAR Academy
Boost your skills with the Netgear Academy - Get trained, certified and stay ahead with the latest Netgear technology!
Join Us!