OS X and Linux both use the same line endings, so I don't think you will have a problem unless you're transferring to a Windoze server.eyoungren wrote:For OSX, the default for Fetch and Cyberduck is Automatic. Don't know what it is for Transmit. Cyberduck seems to handle automatic much better than Fetch, but I've switched it to Binary anyway.
Yes.eyoungren wrote:Does this apply to SFTP?
No, I use the ftp command in Mac OS X, and it defaults to binary. In fact, when I know that I'm just uploading ascii files, I type ascii to switch to that mode.Noxwizard wrote: Does your FTP client do this by default too?
Code: Select all
230 OK. Current restricted directory is /
Remote system type is UNIX.
Using binary mode to transfer files.
ftp> ascii
200 TYPE is now ASCII
ftp>
No.Have you been burned by this "feature" before?
SFTP operates over an SSH connection. It makes no discrimination between ascii or binary modes.eyoungren wrote:Cyberduck does not seem to have any switches other than the type of security transfer.
Does this apply to SFTP?
There is absolutely a wrong method. One method destroys stuff, the other method doesn't.updown wrote:This seems to be some kind of "religious" problem, depending on what each developer believes in. There is no absolute right or wrong, and I doubt that there will be a unique standard, as long as the CRLF-LF-differences themselves exist due to the different systems.
This is actually easy to answer by reading RFC 959:How an FTP client can assume that a completely unknown file is going to be pure ASCII is beyond me.
Curiously enough,3.1.1.1. ASCII TYPE
This is the default type and must be accepted by all FTP
implementations.
So, using binary mode by default will render an implementation non-compliant with the specification, and possibly non-interoperable with other implementations.3.1.1.3. IMAGE TYPE
Image type is intended for the efficient storage and
retrieval of files and for the transfer of binary data. It
is recommended that this type be accepted by all FTP
implementations.
I wanted to explore that a bit. I was under the impression that going from Unix to Windows would convert every LF (including those in CRLF pairs already) to CRLF. If that were the case, reuploading as ASCII would be OK (subject to the following) as any existing CRLF pairs that got translated to CRCRLF sequences as part of the DOS conversion would just get converted back to CRLF.NoxWizard wrote:Recovering the files is not possible at this point. The only option is to change the transfer mode and download them again if you still have access to the originals. “But can’t I just re-upload the files and let the line endings get converted back?” Unfortunately, no. Existing Line Feeds would have had a Carriage Return added in front of them, and existing CRLFs would have been left alone. Uploading it would convert both the new and the previous CRLFs to LF. The file is lost.
Interestingly, that RFC defined end-of-line as:Oleg wrote:This is actually easy to answer by reading RFC 959[...]How an FTP client can assume that a completely unknown file is going to be pure ASCII is beyond me.
So it looks like UNIX got it wrong. Interpreting the character names (Carriage Return and Line Feed) literally, a carriage return without a line feed should allow overtyping (on a printing terminal; it's more difficult with a monitor ) and a line feed without a carriage return should start printing on the next line in the next horizontal character position (ragged text).End-of-Line
The end-of-line sequence defines the separation of printing
lines. The sequence is Carriage Return, followed by Line Feed.
Yes, I've read it, but that's really only a valid assumption if the client has decided to implement that single data type. If you're offering both and a mode called "Automatic" that is supposed to do things magically without breaking files, it should try to be intelligent and not cause data corruption. When in doubt, no transformations should be applied. If the server or client only offer one of the modes, then there is no choice but to treat it as ASCII.Oleg wrote:This is actually easy to answer by reading RFC 959:How an FTP client can assume that a completely unknown file is going to be pure ASCII is beyond me.3.1.1.1. ASCII TYPE
This is the default type and must be accepted by all FTP
implementations.
You have tried to make that recommendation before and were told the same thing. Try it yourself, it only takes a minute. Additionally, there is no EOF character embedded in files in Windows. By your own article, it is a concept and not an actual character. Anyway, the transformations only apply to line endings:Pony99CA wrote:I wanted to explore that a bit. I was under the impression that going from Unix to Windows would convert every LF (including those in CRLF pairs already) to CRLF. If that were the case, reuploading as ASCII would be OK (subject to the following) as any existing CRLF pairs that got translated to CRCRLF sequences as part of the DOS conversion would just get converted back to CRLF.
I thought the real problem would be the DOS End-of-File character. If ASCII mode recognizes those, a binary file that happened to contain '1A'X would get truncated when converting to ASCII. That would be truly unrecoverable.
Steve
3.4. TRANSMISSION MODES wrote:For the purpose of standardized transfer, the sending host will
translate its internal end of line or end of record denotation
into the representation prescribed by the transfer mode and file
structure, and the receiving host will perform the inverse
translation to its internal denotation.
This is how the FTP RFC defines it for its own use, not how everyone else should use it.Pony99CA wrote:So it looks like UNIX got it wrong.
A number of (popular) protocols (including HTTP and SMTP) define "end of line" as CR+LF. I don't know why this is.Pony99CA wrote: So it looks like UNIX got it wrong.
I agree that a client is free to use any mode it is able to negotiate with a server.Noxwizard wrote: Yes, I've read it, but that's really only a valid assumption if the client has decided to implement that single data type. If you're offering both and a mode called "Automatic" that is supposed to do things magically without breaking files, it should try to be intelligent and not cause data corruption. When in doubt, no transformations should be applied. If the server or client only offer one of the modes, then there is no choice but to treat it as ASCII.
First, my post here was not a "recommendation"; I was seeking clarification that CRLF in a UNIX file did not get translated to CRCRLF during an ASCII transfer to Windows.Noxwizard wrote:You have tried to make that recommendation before and were told the same thing. Try it yourself, it only takes a minute.Pony99CA wrote:I wanted to explore that a bit. I was under the impression that going from Unix to Windows would convert every LF (including those in CRLF pairs already) to CRLF. If that were the case, reuploading as ASCII would be OK (subject to the following) as any existing CRLF pairs that got translated to CRCRLF sequences as part of the DOS conversion would just get converted back to CRLF.
I thought the real problem would be the DOS End-of-File character. If ASCII mode recognizes those, a binary file that happened to contain '1A'X would get truncated when converting to ASCII. That would be truly unrecoverable.
Code: Select all
01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0D 0A
01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0D 0A
1A 1A 1A 46 54 50
Code: Select all
01 02 03 04 05 06 07 08 09 0d 0a 0b 0c 0d 0e 0d
0d 0a 01 02 03 04 05 06 07 08 09 0d 0a 0b 0c 0d
0e 0d 0d 0a 1a 1a 1a 46 54 50
Code: Select all
01 02 03 04 05 06 07 08 09 0a 0b 0c 0e 0a 01 02
03 04 05 06 07 08 09 0a 0b 0c 0e 0a 1a 1a 1a 46
54 50
Did we read the same article? The one that I linked to clearly stated that DOS used Ctrl+Z as an End-of-File character (for compatibility with CP/M and to enable file input and terminal input to be handled identically). Maybe Windows no longer uses Ctrl+Z, but that's a different discussion.Noxwizard wrote:Additionally, there is no EOF character embedded in files in Windows. By your own article, it is a concept and not an actual character.
Remind me next time to put more smilies in there to indicate when something is intended as a joke.Noxwizard wrote:This is how the FTP RFC defines it for its own use, not how everyone else should use it.Pony99CA wrote:So it looks like UNIX got it wrong.