Gordon,
Date: Sat, 20 Feb 2016 16:40:01 -0800 From: Gordon Messmer gordon.messmer@gmail.com To: Community support for Fedora users users@lists.fedoraproject.org Subject: Re: Bash / Escaping quotes is driving me crazy . . Message-ID: 56C90761.4070509@gmail.com Content-Type: text/plain; charset=utf-8; format=flowed
On 02/20/2016 04:05 PM, Philip Rhoades wrote:
. . but why is there only a problem with the "flac" OR? - all three files have at least one space in the filename:
Your mistake seems to be believing that the shell can understand the way you're nesting quotes. It can't. Each unescaped quote you're using simply terminates the quoted string that preceded it. So your example:
ssh localhost "find /home/phil/music/ambient/RobertGass+OnWingsOfSong/OmNamahaShivaya -maxdepth 1 -type f \( -name "*.mp3" -o -name "*.m4a" -o -name "*.flac" \)"
There are the following quoted strings:
"find /home/phil/music/ambient/RobertGass+OnWingsOfSong/OmNamahaShivaya -maxdepth 1 -type f \( -name " " -o -name " " -o -name " " \)"
This means two things: First, the wildcards are unquoted when the command is run on the remote system. Second, the wildcards are also unquoted on the local system. So, if there are any mp3, m4a, or flac files in the directory where you run that command, or in the default login directory on the remote system, the wildcard will be expanded and find will only search for files with that specific name.
Instead, use single quotes around the entire command, or escape both the quotes and the wildcards.
ssh localhost "find /home/phil/music/ambient/RobertGass+OnWingsOfSong/OmNamahaShivaya -maxdepth 1 -type f \( -name "*.mp3" -o -name "*.m4a" -o -name "*.flac" \)"
ssh localhost 'find /home/phil/music/ambient/RobertGass+OnWingsOfSong/OmNamahaShivaya -maxdepth 1 -type f ( -name "*.mp3" -o -name "*.m4a" -o -name "*.flac" )'
OK, that all makes sense but there is a further issue - I was trying to keep it simple - this whole line is inside a Ruby "system" command ie:
system( "ssh .. " )
- so my working version is the same as your first option (without the escaped '*'s) but with an extra '' at each place. I can't use the second option because I need to use double quotes so that I can use Ruby variables inside the double quotes eg:
#{path}
Thanks for the explanation!
Regards,
Phil.
On 02/20/2016 07:18 PM, Philip Rhoades wrote:
OK, that all makes sense but there is a further issue - I was trying to keep it simple - this whole line is inside a Ruby "system" command ie: system( "ssh .. " ) I can't use the second option because I need to use double quotes so that I can use Ruby variables inside the double quotes eg:
I think you're still missing some fundamental concepts about nesting quotes. (I was also mistaken in suggesting that the wildcards needed escaping when the internal double-quotes were escaped, though, so... We all make mistakes.)
In Ruby double-quoted strings, you can get interpolation, and in single quoted strings you don't. But if you nest single quotes inside a double-quoted strings, Ruby still treats the entire string as double quoted. It doesn't change the rules when it finds single quotes inside the double-quoted string, because they're merely a part of the double-quoted string. So your options are:
system("ssh localhost "find /home/... -maxdepth 1 -type f \\( -name \"*.mp3\" -o -name \"*.m4a\" -o -name \"*.flac\" \\)" ")
or:
system("ssh localhost 'find /home/... -maxdepth 1 -type f \( -name "*.mp3" -o -name "*.m4a" -o -name "*.flac" \)' ")
Using single quotes means significantly less escaping, and you can still use interpolation in any part of that string, in Ruby.
On 21Feb2016 14:18, Philip Rhoades phil@pricom.com.au wrote:
Date: Sat, 20 Feb 2016 16:40:01 -0800 From: Gordon Messmer gordon.messmer@gmail.com
[...]
On 02/20/2016 04:05 PM, Philip Rhoades wrote:
. . but why is there only a problem with the "flac" OR? - all three files have at least one space in the filename:
Your mistake seems to be believing that the shell can understand the way you're nesting quotes. It can't. Each unescaped quote you're using simply terminates the quoted string that preceded it. So your example:
ssh localhost "find /home/phil/music/ambient/RobertGass+OnWingsOfSong/OmNamahaShivaya -maxdepth 1 -type f \( -name "*.mp3" -o -name "*.m4a" -o -name "*.flac" \)"
Because quoting shell commands to dispatch over ssh is tedious, I have this script:
https://bitbucket.org/cameron_simpson/css/src/tip/bin/sshx
which accepts an unquoted command line and does the work for you, so your request would be written:
sshx localhost find /your/music/dir -maxdepth 1 -type f ( -name *.mp3 -o -name *.m4a -o -name *.flac
i.e. exactly as you would without the leading "ssh localhost" to an shell prompt.
Please feel free to fetch it (use the "Raw" link at top right) and use it. It saves a lot of pain.
BTW, why "localhost"? Or is this just an example for test, to be used on other hosts later when working?
OK, that all makes sense but there is a further issue - I was trying to keep it simple - this whole line is inside a Ruby "system" command ie:
system( "ssh .. " )
- so my working version is the same as your first option (without the
escaped '*'s) but with an extra '' at each place. I can't use the second option because I need to use double quotes so that I can use Ruby variables inside the double quotes eg:
#{path}
All this means is that you need to take your shell command line, once you have a working one, and quote it again to express it correctly as a Ruby string. Bear with me, because I don't yet speak Ruby.
SO first get your shell command line. Make it as simple as feasible. If you've fetched my "sshx" script, that can be:
sshx localhost find ...
quoted no more than any local "find" command would be. Now express that as a Ruby string:
https://en.wikibooks.org/wiki/Ruby_Programming/Syntax/Literals#Strings
Since you're using double quotes to allow # interpolation (this is not a great practive for command lines, to which we'll come later) you probably want to either run with a command line with no double quotes in it and with all the backslashes doubled:
shcmd = "sshx localhost find /dir -name \*.mp3 ..."
If _are_ using a command line with double quotes, they can also be escaped, eg:
shcmd = "sshx localhost find /dir -name "*.mp3" ..."
I'm imagining that you're expecting to replace "localhost" or the directory with something like #{remotehost} and #{path} later, thus:
shcmd = "sshx #{remotehost} find #{path} -name "*.mp3" ..."
and here we come to the issue of quoting yet again. Supposing your path has spaces or other shell punctuation in it. You also need to quote it, _before_ using it in your command line string.
I find it useful to have a "shell quote" function in my kit, whatever the language. You can take any string to be used in the shell undamaged and suitable quote it by replacing all single quotes with:
'''
and enclosing the whole result in a pair of single quotes. You would need to do this for the variable you want to interpolate:
qpath = shqstr(path) qremotehost = shqstr(remotehost)
and then interpolate _those_ values:
shcmd = "sshx #{qremotehost} find #{qpath} -name "*.mp3" ..."
and finally running:
system(shcmd)
However, it is better to avoid going through the shell at all if it is only an intermediary to running a single command (your single "ssh" command). Using system() with a carefully quoted string has all the same fiddliness as passing carefully constructed SQL to a database query: any tiny flaw in the quoting can lead to incorrect behaviour, and if any of the strings come from user input, that is also an avenue for attack (there are known as injection attacks, where use input is injected into a command you issue, perverting your intent):
The URL above uses SQL because it is an incredibly command misuse, but htis issue applies equally well to shell commands constructed the way you are constructing them.
To avoid going though the shell at all you want to invoke your command with the Subprocess Ruby module:
http://www.rubydoc.info/github/stripe/subprocess/Subprocess
Construct your command as a list of strings:
cmd = ['sshx', remotehost, 'find', path, '-name', '*.mp3', ...]
and invoke Subprocess.popen to collect the resulting output. Notice that there's no quoting there at all, including of things like *.mp3. This is because the shell will never see these strings, and therefore you need only present them directly as valid Ruby strings.
Cheers, Cameron Simpson cs@zip.com.au
On Sun, Feb 21, 2016 at 02:18:45PM +1100, Philip Rhoades wrote:
OK, that all makes sense but there is a further issue - I was trying to keep it simple - this whole line is inside a Ruby "system" command ie:
Well that's pretty frightening. :)
Have you considered using ruby's Net::SSH instead?
-
Cameron Simpson -
Gordon Messmer -
Matthew Miller -
Philip Rhoades