8:44 a.m.
Hi Fedora users!
Over the last years, there were several issues in the SCP protocol,
which lead us into discussions if we can get rid of it in upstream [1].
Most of the voices there said that they use SCP mostly for simple ad-hoc
copy and because sftp utility does not provide simple interface to copy
one or couple of files back and forth and because of people are just
used to write scp rather than sftp.
Some months ago, I wrote a patch [2] for scp to use SFTP internally
(with possibility to change it back using -M scp) and ran it through
some successful testing. The general feedback from upstream was also
quite positive so I would like to hear also opinions from our users.
It still has some limitations (missing -3 support, it will not work if
the server does not run sftp subsystem, ...), but it should be good
enough for most common use cases.
Today, I set up a copr repository with the current openssh from Fedora +
the patch [2] for anyone to test and provide feedback, either here on
the mailing list, or in the github PR according to ones preferences.
I am looking for any kind of feedback from the idea through the
usability, implementation. Is this something you would like to see in
Fedora soon? Do you have something against this? Is your use case missing?
[1]
https://lists.mindrot.org/pipermail/openssh-unix-dev/2020-June/038594.html
[2] https://github.com/openssh/openssh-portable/pull/194/
[3] https://copr.fedorainfracloud.org/coprs/jjelen/openssh-sftp/
Thanks,
--
Jakub Jelen
Senior Software Engineer
Crypto Team, Security Engineering
Red Hat, Inc.
8:57 a.m.
New subject: [Fedora] Deprecating SCP
Hi,
I don't know how if and how the internet protocol scp: is related to the scp
command. But I suppose it is.
I'm using scp: a lot to edit remote files with vim and I'm pretty sure that
many remote admins are doing the same.
So I'm wondering how this change will affect my use case scenario and if you
have considered it when moving to sftp.
Thank you
Walter
On Mon, 2 Nov 2020, Jakub Jelen wrote:
Hi Fedora users!
Over the last years, there were several issues in the SCP protocol, which
lead us into discussions if we can get rid of it in upstream [1]. Most of the
voices there said that they use SCP mostly for simple ad-hoc copy and because
sftp utility does not provide simple interface to copy one or couple of files
back and forth and because of people are just used to write scp rather than
sftp.
Some months ago, I wrote a patch [2] for scp to use SFTP internally (with
possibility to change it back using -M scp) and ran it through some
successful testing. The general feedback from upstream was also quite
positive so I would like to hear also opinions from our users.
It still has some limitations (missing -3 support, it will not work if the
server does not run sftp subsystem, ...), but it should be good enough for
most common use cases.
Today, I set up a copr repository with the current openssh from Fedora + the
patch [2] for anyone to test and provide feedback, either here on the mailing
list, or in the github PR according to ones preferences.
I am looking for any kind of feedback from the idea through the usability,
implementation. Is this something you would like to see in Fedora soon? Do
you have something against this? Is your use case missing?
[1]
https://lists.mindrot.org/pipermail/openssh-unix-dev/2020-June/038594.html
[2] https://github.com/openssh/openssh-portable/pull/194/
[3] https://copr.fedorainfracloud.org/coprs/jjelen/openssh-sftp/
Thanks,
--
9:42 a.m.
New subject: [Fedora] Deprecating SCP
On 11/2/20 3:57 PM, Walter Cazzola wrote:
Hi,
I don't know if and how the internet protocol scp: is related to the scp
command. But I suppose it is.
Hi,
SCP is not an internet protocol -- it is simple protocol that is used
inside of encrypted SSH session, similarly to SFTP protocol. The name
comes from RCP which actually was unencrypted internet protocol and
which is hopefully gone.
I'm using scp: a lot to edit remote files with vim and I'm
pretty sure that
many remote admins are doing the same.
So I'm wondering how this change will affect my use case scenario and if
you
have considered it when moving to sftp.
That is a good question!
When I try to use scp://host/file I am getting errors that vim is trying
to use `rcp` command (yuck!). But using the same with sftp://host/file
works like a charm.
I believe vim is using just scp to fetch the file so if the connection
to the server will work also with sftp, it should continue to work (but
I recommend using sftp protocol anyway).
The simplest way to try is to try with sftp:// or try the previously
mentioned package, but my best bet is that it will keep on working as
before (even though I never used this inside of vim up until today).
Regards,
Jakub
Thank you
Walter
On Mon, 2 Nov 2020, Jakub Jelen wrote:
> Hi Fedora users!
>
> Over the last years, there were several issues in the SCP protocol,
> which lead us into discussions if we can get rid of it in upstream
> [1]. Most of the voices there said that they use SCP mostly for simple
> ad-hoc copy and because sftp utility does not provide simple interface
> to copy one or couple of files back and forth and because of people
> are just used to write scp rather than sftp.
>
> Some months ago, I wrote a patch [2] for scp to use SFTP internally
> (with possibility to change it back using -M scp) and ran it through
> some successful testing. The general feedback from upstream was also
> quite positive so I would like to hear also opinions from our users.
>
> It still has some limitations (missing -3 support, it will not work if
> the server does not run sftp subsystem, ...), but it should be good
> enough for most common use cases.
>
> Today, I set up a copr repository with the current openssh from Fedora
> + the patch [2] for anyone to test and provide feedback, either here
> on the mailing list, or in the github PR according to ones preferences.
>
> I am looking for any kind of feedback from the idea through the
> usability, implementation. Is this something you would like to see in
> Fedora soon? Do you have something against this? Is your use case
> missing?
>
> [1]
> https://lists.mindrot.org/pipermail/openssh-unix-dev/2020-June/038594.html
>
> [2] https://github.com/openssh/openssh-portable/pull/194/
> [3] https://copr.fedorainfracloud.org/coprs/jjelen/openssh-sftp/
>
> Thanks,
>
--
Jakub Jelen
Senior Software Engineer
Crypto Team, Security Engineering
Red Hat, Inc.
9:14 a.m.
New subject: [Fedora] Re: [Fedora] Deprecating SCP
On Mon, 2 Nov 2020, Jakub Jelen wrote:
> I'm using scp: a lot to edit remote files with vim and
I'm pretty sure
> that many remote admins are doing the same.
> So I'm wondering how this change will affect my use case
scenario and if
> you have considered it when moving to sftp.
That is a good question!
When I try to use scp://host/file I am getting errors that vim is
trying to
use `rcp` command (yuck!). But using the same with sftp://host/file works
like a charm.
I believe vim is using just scp to fetch the file so if the
connection to the
server will work also with sftp, it should continue to work (but I recommend
using sftp protocol anyway).
The simplest way to try is to try with sftp:// or try the previously
mentioned package, but my best bet is that it will keep on working as before
(even though I never used this inside of vim up until today).
Indeed it does, I wasn't aware that vim supports sftp as well.
Thank you for the prompt reply
Walter
--
9:09 a.m.
On 11/2/20 8:44 AM, Jakub Jelen wrote:
I am looking for any kind of feedback from the idea through the
usability, implementation. Is this something you would like to see in
Fedora soon? Do you have something against this? Is your use case missing?
What impact will this have on compatibility with other operating
systems (Windows 10, etc.)?
--
========================================================================
In Soviet Russia, Google searches you!
========================================================================
2:49 a.m.
On 11/2/20 4:09 PM, Ian Pilcher wrote:
On 11/2/20 8:44 AM, Jakub Jelen wrote:
> I am looking for any kind of feedback from the idea through the
> usability, implementation. Is this something you would like to see in
> Fedora soon? Do you have something against this? Is your use case
> missing?
What impact will this have on compatibility with other operating
systems (Windows 10, etc.)?
If you are able to run sftp server on Windows, you will be able to
connect to it as before.
Regards,
--
Jakub Jelen
Senior Software Engineer
Crypto Team, Security Engineering
Red Hat, Inc.
9:57 a.m.
Hi.
On Mon, 02 Nov 2020 15:44:39 +0100 Jakub Jelen wrote:
Is this something you would like to see in Fedora soon?
No. I prefer a lot to use rsync, because scp:
- has no dry-run mode
- is not incremental
- follows symlinks when used with the -r option
- has too few options: no --chown --chgrp
Do you have something against this?
No: users should be free to continue using it (but not with the -r option IMO).
Is your use case missing?
With scp no, but I use sshfs for years. This is IMO something to promote.
Ex: Example: a simple 3-way copy, assuming you have root SSH access (with keys)
mkdir /mnt/hostA
sshfs -o transform_symlinks root@hostA:/ /mnt/hostA
rsync -a --delete /mnt/hostA/etc/skel/ root@hostB:/etc/skel
--
francis
3:44 a.m.
On 11/2/20 4:57 PM, Francis.Montagnac(a)inria.fr wrote:
Hi.
On Mon, 02 Nov 2020 15:44:39 +0100 Jakub Jelen wrote:
> Is this something you would like to see in Fedora soon?
No. I prefer a lot to use rsync, because scp:
- has no dry-run mode
- is not incremental
- follows symlinks when used with the -r option
- has too few options: no --chown --chgrp
Then you are indeed not the target group.
> Do you have something against this?
No: users should be free to continue using it (but not with the -r option IMO).
The vulnerability of the -r was not the only issue found in scp over
last years. And recursive copying of files is handy so I do not think it
is a good idea to axe just one commandline switch.
> Is your use case missing?
With scp no, but I use sshfs for years. This is IMO something to promote.
Ex: Example: a simple 3-way copy, assuming you have root SSH access (with keys)
mkdir /mnt/hostA
sshfs -o transform_symlinks root@hostA:/ /mnt/hostA
rsync -a --delete /mnt/hostA/etc/skel/ root@hostB:/etc/skel
sshfs is using sftp internally so you are already using the sftp.
Congratulations.
Regards,
--
Jakub Jelen
Senior Software Engineer
Crypto Team, Security Engineering
Red Hat, Inc.
10 a.m.
On 02/11/2020 15:44, Jakub Jelen wrote:
Hi Fedora users!
Over the last years, there were several issues in the SCP protocol, which lead
us into discussions if we can get rid of it in upstream [1]. Most of the voices
there said that they use SCP mostly for simple ad-hoc copy and because sftp
utility does not provide simple interface to copy one or couple of files back
and forth and because of people are just used to write scp rather than sftp.
Some months ago, I wrote a patch [2] for scp to use SFTP internally (with
possibility to change it back using -M scp) and ran it through some successful
testing. The general feedback from upstream was also quite positive so I would
like to hear also opinions from our users.
[..]
In July there was an article in Fedoramagazine about switching from scp to rsync
as "the OpenSSH project stated that they consider the scp protocol outdated,
inflexible, and not readily fixed".
In fact for the simple use "rsync FILE dest:DIR" instead of "scp FILE
dest:DIR"
is easy to do.
I suppose it's time....
G
P.S. Sorry for the mail Jakub, I did a "reply" instead of "Reply
List"
--
GianPiero Puccioni |Istituto dei Sistemi Complessi-CNR
gianpiero.puccioni(a)isc.cnr.it |Via Madonna del Piano, 10
T:+39 0555226682 |50019 Sesto F. (Firenze) ITALY
10:04 a.m.
On Mon, Nov 02, 2020 at 03:44:39PM +0100, Jakub Jelen wrote:
Over the last years, there were several issues in the SCP protocol,
which
lead us into discussions if we can get rid of it in upstream [1]. Most of
the voices there said that they use SCP mostly for simple ad-hoc copy and
because sftp utility does not provide simple interface to copy one or couple
of files back and forth and because of people are just used to write scp
rather than sftp.
Some months ago, I wrote a patch [2] for scp to use SFTP internally (with
possibility to change it back using -M scp) and ran it through some
successful testing. The general feedback from upstream was also quite
positive so I would like to hear also opinions from our users.
Has that testing included performance measurements, both on high bandwidth
low-latency transfers and low bandwidth high-latency transfers?
At least in the past SFTP used to be worse than SCP on high-latency
connections.
Jakub
10:19 a.m.
On Mon, 2 Nov 2020 15:44:39 +0100
Jakub Jelen wrote:
Do you have something against this?
I use the scp command all the time, if the command is still there
I don't care if it does something different under the hood.
I suppose I could always use rsync instead of the command
disappeared.
7:04 a.m.
On Tue, 3 Nov 2020 at 12:20, Tom Horsley <horsley1953(a)gmail.com> wrote:
On Mon, 2 Nov 2020 15:44:39 +0100
Jakub Jelen wrote:
> Do you have something against this?
I use the scp command all the time, if the command is still there
I don't care if it does something different under the hood.
I suppose I could always use rsync instead of the command
disappeared.
I doubt most people will notice the difference [until some scp
download installs malware].
PuTTy (Windows) scp has been using sftp when the remote
server supports it for a couple of years. In my field, many users
(due to enterprise desktop standards) get Windows workstations
Linux HPC does the heavy lifting, so pscp is often used to download
artifacts.
https://www.ssh.com/ssh/putty/putty-manuals/0.68/Chapter5.html
This is due to a fundamental insecurity in the old-style SCP protocol: the
client sends the wildcard string (*.c) to the server, and the server sends
back a sequence of file names that match the wildcard pattern. However,
there is nothing to stop the server sending back a different pattern and
writing over one of your other files: if you request *.c, the server might
send back the file name AUTOEXEC.BAT and install a virus for you. Since the
wildcard matching rules are decided by the server, the client cannot
reliably verify that the filenames sent back match the pattern.
PSCP will attempt to use the newer SFTP protocol (part of SSH-2) where
possible, which does not suffer from this security flaw. If you are talking
to an SSH-2 server which supports SFTP, you will never see this warning.
(You can force use of the SFTP protocol, if available, with -sftp - see
section 5.2.2.6.)
--
George N. White III
7:40 a.m.
On Wed, Nov 4, 2020 at 6:34 PM George N. White III <gnwiii(a)gmail.com> wrote:
On Tue, 3 Nov 2020 at 12:20, Tom Horsley
<horsley1953(a)gmail.com> wrote:
> On Mon, 2 Nov 2020 15:44:39 +0100
> Jakub Jelen wrote:
>
> > Do you have something against this?
>
> I use the scp command all the time, if the command is still there
> I don't care if it does something different under the hood.
> I suppose I could always use rsync instead of the command
> disappeared.
>
I doubt most people will notice the difference [until some scp
download installs malware].
PuTTy (Windows) scp has been using sftp when the remote
server supports it for a couple of years. In my field, many users
(due to enterprise desktop standards) get Windows workstations
Linux HPC does the heavy lifting, so pscp is often used to download
artifacts.
https://www.ssh.com/ssh/putty/putty-manuals/0.68/Chapter5.html
This is due to a fundamental insecurity in the old-style SCP protocol: the
client sends the wildcard string (*.c) to the server, and the server sends
back a sequence of file names that match the wildcard pattern. However,
there is nothing to stop the server sending back a different pattern and
writing over one of your other files: if you request *.c, the server might
send back the file name AUTOEXEC.BAT and install a virus for you. Since the
wildcard matching rules are decided by the server, the client cannot
reliably verify that the filenames sent back match the pattern.
PSCP will attempt to use the newer SFTP protocol (part of SSH-2) where
possible, which does not suffer from this security flaw. If you are talking
to an SSH-2 server which supports SFTP, you will never see this warning.
(You can force use of the SFTP protocol, if available, with -sftp - see
section 5.2.2.6.)
--
George N. White III
rsync is not installed by default in minimal installs of many OS.
For example, we
have to scp the internet login script to the new OS and
then install rsync after internet login.
---
Lee
9:39 a.m.
I've not used sftp at all.
I've only used scp when I had control at both ends.
On Wed, 4 Nov 2020, George N. White III wrote:
This is due to a fundamental insecurity in the old-style SCP
protocol: the
client sends the wildcard string (*.c) to the server, and the server sends
back a sequence of file names that match the wildcard pattern. However,
there is nothing to stop the server sending back a different pattern and
writing over one of your other files: if you request *.c, the server might
send back the file name AUTOEXEC.BAT and install a virus for you. Since the
wildcard matching rules are decided by the server, the client cannot
reliably verify that the filenames sent back match the pattern.
In this particular case, I'd think the client
could tell that a .BAT file was not a .c file.
There is only so much any protocol can do about a malicious server.
Even if the client explicitly specifies a server file,
there is no guarantee that the server will send a correct copy.
Note: no quoted boilerplate. Please emulate.
--
Michael hennebry(a)web.cs.ndsu.NoDak.edu
"Sorry but your password must contain an uppercase letter, a number,
a haiku, a gang sign, a heiroglyph, and the blood of a virgin."
-- someeecards
11:44 a.m.
On Wed, 4 Nov 2020 at 11:39, Michael Hennebry <
hennebry(a)web.cs.ndsu.nodak.edu> wrote:
I've not used sftp at all.
I've only used scp when I had control at both ends.
On Wed, 4 Nov 2020, George N. White III wrote:
> This is due to a fundamental insecurity in the old-style SCP protocol:
the
> client sends the wildcard string (*.c) to the server, and the server
sends
> back a sequence of file names that match the wildcard pattern. However,
> there is nothing to stop the server sending back a different pattern and
> writing over one of your other files: if you request *.c, the server
might
> send back the file name AUTOEXEC.BAT and install a virus for you. Since
the
> wildcard matching rules are decided by the server, the client cannot
> reliably verify that the filenames sent back match the pattern.
In this particular case, I'd think the client
could tell that a .BAT file was not a .c file.
Downloading 1000's of files resulting from some HPC calculation
it would be easy to overlook an unwanted file.
There is only so much any protocol can do about a malicious server.
Even if the client explicitly specifies a server file,
there is no guarantee that the server will send a correct copy.
Those HPC systems can have many users, all with write access to
shared data spaces; one compromised user workstation could be
used to place malware where scp might find it. It is more of a
concern now that user workstations have moved into people's
homes outside the enterprise LAN's.
--
George N. White III
12:49 p.m.
On Wed, 4 Nov 2020, George N. White III wrote:
On Wed, 4 Nov 2020 at 11:39, Michael Hennebry <
hennebry(a)web.cs.ndsu.nodak.edu> wrote:
> In this particular case, I'd think the client
> could tell that a .BAT file was not a .c file.
>
Downloading 1000's of files resulting from some HPC calculation
it would be easy to overlook an unwanted file.
To be clear, I'd meant the client could to it automatically.
--
Michael hennebry(a)web.cs.ndsu.NoDak.edu
"Sorry but your password must contain an uppercase letter, a number,
a haiku, a gang sign, a heiroglyph, and the blood of a virgin."
-- someeecards
5:31 a.m.
On 11/5/20 7:49 PM, Michael Hennebry wrote:
On Wed, 4 Nov 2020, George N. White III wrote:
> On Wed, 4 Nov 2020 at 11:39, Michael Hennebry <
> hennebry(a)web.cs.ndsu.nodak.edu> wrote:
>> In this particular case, I'd think the client
>> could tell that a .BAT file was not a .c file.
>>
>
> Downloading 1000's of files resulting from some HPC calculation
> it would be easy to overlook an unwanted file.
To be clear, I'd meant the client could to it automatically.
To be clear, the scp client does finename matching already by default.
But if there is a recursive download (or a regex matching the malicious
file), this might not help either.
Not even using sftp, if the file is sneaked inside of the downloaded
directory. But breaking scp protocol to do that is a matter of inserting
one command, while SFTP requires significantly more work and the
downloads can be properly audited (unlike scp, which is just a command
on the server).
Regards,
--
Jakub Jelen
Senior Software Engineer
Crypto Team, Security Engineering
Red Hat, Inc.
1322
days inactive
1330
days old
16 comments
10 participants
participants (10)
-
Francis.Montagnac＠inria.fr -
George N. White III -
GianPiero Puccioni -
Ian Pilcher -
Jakub Jelen -
Jakub Jelinek -
Michael Hennebry -
Thomas Stephen Lee -
Tom Horsley -
Walter Cazzola