Bash / Escaping quotes is driving me crazy . .

Tue Feb 23 01:51:45 UTC 2016

Cameron,

> Date: Tue, 23 Feb 2016 08:39:07 +1100
> From: Cameron Simpson <cs at zip.com.au>
> To: phil at pricom.com.au,	Community support for Fedora users
> 	<users at lists.fedoraproject.org>
> Subject: Re: Bash / Escaping quotes is driving me crazy . .
> Message-ID: <20160222213907.GA79479 at cskk.homeip.net>
> Content-Type: text/plain; charset=us-ascii; format=flowed
> 
> On 21Feb2016 14:18, Philip Rhoades <phil at pricom.com.au> wrote:
>>> Date: Sat, 20 Feb 2016 16:40:01 -0800
>>> From: Gordon Messmer <gordon.messmer at gmail.com>
> [...]
>>> On 02/20/2016 04:05 PM, Philip Rhoades wrote:
>>>> . . but why is there only a problem with the "flac" OR? - all three
>>>> files have at least one space in the filename:
>>> 
>>> Your mistake seems to be believing that the shell can understand the
>>> way
>>> you're nesting quotes.  It can't.  Each unescaped quote you're using
>>> simply terminates the quoted string that preceded it.  So your 
>>> example:
>>> 
>>>   ssh localhost "find
>>> /home/phil/music/ambient/RobertGass+OnWingsOfSong/OmNamahaShivaya
>>> -maxdepth 1 -type f \\( -name "*.mp3" -o -name "*.m4a" -o -name
>>> "*.flac"
>>> \\)"
> 
> Because quoting shell commands to dispatch over ssh is tedious, I have 
> this
> script:
> 
>   https://bitbucket.org/cameron_simpson/css/src/tip/bin/sshx
> 
> which accepts an unquoted command line and does the work for you, so 
> your
> request would be written:
> 
>   sshx localhost find /your/music/dir -maxdepth 1 -type f \( -name
> \*.mp3 -o -name \*.m4a -o -name \*.flac
> 
> i.e. exactly as you would without the leading "ssh localhost" to an 
> shell
> prompt.
> 
> Please feel free to fetch it (use the "Raw" link at top right) and use 
> it. It
> saves a lot of pain.

Excellent!  Thanks for that!

> BTW, why "localhost"? Or is this just an example for test, to be used 
> on other
> hosts later when working?

Exactly.

>> OK, that all makes sense but there is a further issue - I was trying 
>> to keep
>> it simple - this whole line is inside a Ruby "system" command ie:
>> 
>>  system( "ssh .. " )
>> 
>> - so my working version is the same as your first option (without the
>> escaped '*'s) but with an extra '\' at each place.  I can't use the
>> second option because I need to use double quotes so that I can use
>> Ruby variables inside the double quotes eg:
>> 
>>  #{path}
> 
> All this means is that you need to take your shell command line, once 
> you have
> a working one, and quote it again to express it correctly as a Ruby 
> string.
> Bear with me, because I don't yet speak Ruby.
> 
> SO first get your shell command line. Make it as simple as feasible. If 
> you've
> fetched my "sshx" script, that can be:
> 
>   sshx localhost find ...
> 
> quoted no more than any local "find" command would be. Now express that 
> as a
> Ruby string:
> 
>   
> https://en.wikibooks.org/wiki/Ruby_Programming/Syntax/Literals#Strings
> 
> Since you're using double quotes to allow # interpolation (this is not 
> a great
> practive for command lines, to which we'll come later) you probably 
> want to
> either run with a command line with no double quotes in it and with all 
> the
> backslashes doubled:
> 
>   shcmd = "sshx localhost find /dir -name \\*.mp3 ..."
> 
> If _are_ using a command line with double quotes, they can also be 
> escaped, eg:
> 
>   shcmd = "sshx localhost find /dir -name \"*.mp3\" ..."
> 
> I'm imagining that you're expecting to replace "localhost" or the 
> directory
> with something like #{remotehost} and #{path} later, thus:
> 
>   shcmd = "sshx #{remotehost} find #{path} -name \"*.mp3\" ..."
> 
> and here we come to the issue of quoting yet again. Supposing your path 
> has
> spaces or other shell punctuation in it. You also need to quote it, 
> _before_
> using it in your command line string.
> 
> I find it useful to have a "shell quote" function in my kit, whatever 
> the
> language. You can take any string to be used in the shell undamaged and
> suitable quote it by replacing all single quotes with:
> 
>   '\''
> 
> and enclosing the whole result in a pair of single quotes. You would 
> need to do
> this for the variable you want to interpolate:
> 
>   qpath = shqstr(path)
>   qremotehost = shqstr(remotehost)
> 
> and then interpolate _those_ values:
> 
>   shcmd = "sshx #{qremotehost} find #{qpath} -name \"*.mp3\" ..."
> 
> and finally running:
> 
>   system(shcmd)
> 
> However, it is better to avoid going through the shell at all if it is 
> only an
> intermediary to running a single command (your single "ssh" command). 
> Using
> system() with a carefully quoted string has all the same fiddliness as 
> passing
> carefully constructed SQL to a database query: any tiny flaw in the 
> quoting can
> lead to incorrect behaviour, and if any of the strings come from user 
> input,
> that is also an avenue for attack (there are known as injection 
> attacks, where
> use input is injected into a command you issue, perverting your 
> intent):
> 
>   http://bobby-tables.com/
> 
> The URL above uses SQL because it is an incredibly command misuse, but 
> htis
> issue applies equally well to shell commands constructed the way you 
> are
> constructing them.
> 
> To avoid going though the shell at all you want to invoke your command 
> with the
> Subprocess Ruby module:
> 
>   http://www.rubydoc.info/github/stripe/subprocess/Subprocess
> 
> Construct your command as a list of strings:
> 
>   cmd = ['sshx', remotehost, 'find', path, '-name', '*.mp3', ...]
> 
> and invoke Subprocess.popen to collect the resulting output. Notice 
> that
> there's no quoting there at all, including of things like *.mp3. This 
> is
> because the shell will never see these strings, and therefore you need 
> only
> present them directly as valid Ruby strings.

Wow!  Thanks for taking the time to answer in such useful detail!  I 
will have a look at all that - interesting stuff!

And thanks to everyone else who helped as well.

Regards,

Phil.
-- 
Philip Rhoades

PO Box 896
Cowra  NSW  2794
Australia
E-mail:  phil at pricom.com.au