Couldn't we enable 256 colors by default on TERM?

[Kévin Raymond]

Hi there,

We are building a leading edge Operating System, but still use only 8bit colors
by default in our terminal (I don't know about KDE… I stay under
GNOME, gnome-term (xterm)).
This limit the colors of many applications like vim, screen, tmux, weechat…

As seen (quick but not exhaustive check), all dependencies for xterm
in 256 colors are there,
we just need to define `TERM=xterm-256color' in order to get 256 colors…

So what would be needed?


-- 
Kévin Raymond
(shaiton)
GPG-Key: A5BCB3A2

[Pádraig Brady]

On 05/30/2012 09:30 AM, Kévin Raymond wrote:
>  Hi there,
>  
>  We are building a leading edge Operating System, but still use only 8bit colors
>  by default in our terminal (I don't know about KDE… I stay under
>  GNOME, gnome-term (xterm)).
>  This limit the colors of many applications like vim, screen, tmux, weechat…
>  
>  As seen (quick but not exhaustive check), all dependencies for xterm
>  in 256 colors are there,
>  we just need to define `TERM=xterm-256color' in order to get 256 colors…
>  
>  So what would be needed? 
I've some notes about 256 colors here:
http://www.pixelbeat.org/docs/terminal_colours/#256

If you restrict yourself to local xterms
(of most varieties) you're fine.

However...

256 color doesn't work correctly on Linux virtual consoles.

Also since your $TERM is propagated to remote systems
when you ssh, if they don't support that you'll have issues.
For example sshing to debian systems (I've not tried newer releases)
will be problematic unless a package in installed there.
Also if I ssh to solaris then I need to reset TERM to "xterm"
for man pages to work correctly etc.

So it's one of those borderline things that is unfortunately
dependent on the users environment, so I'd be wary about
changing this default.

cheers,
Pádraig.

[Matthew Garrett]

We discussed this in fesco today and had a couple of concerns. The 
primary one was that handling this in profile seemed a bit fragile. It 
seems like it would be more "correct" to have the terminals explicitly 
set xterm-256 themselves if they're capable of it, rather than assuming 
things about the binaries that a user may have installed. It's a little 
more work, although not a great deal - by the looks of it vte sets this 
itself, so a single patch would handle most of the GTK cases.

Thoughts?
-- 
Matthew Garrett | mjg59(a)srcf.ucam.org

[Chris Adams]

Once upon a time, Matthew Garrett <mjg59(a)srcf.ucam.org&gt; said:
>  We discussed this in fesco today and had a couple of concerns. The 
>  primary one was that handling this in profile seemed a bit fragile. It 
>  seems like it would be more "correct" to have the terminals explicitly 
>  set xterm-256 themselves if they're capable of it, rather than assuming 
>  things about the binaries that a user may have installed. It's a little 
>  more work, although not a great deal - by the looks of it vte sets this 
>  itself, so a single patch would handle most of the GTK cases.
>  
>  Thoughts? 
That would make a lot more sense to me - TERM is set by the terminal
program to communicate its functionality to the shell and child
programs.  The shell init (profile) scripts should not change that
because they "know better"; if the terminal supports 256 colors, it
should set an appropriate TERM value to indicate that.

You should basically _never_ override TERM unless you really know what
you are doing.  For example, when SSHing to something with a smaller
terminal info database, you might override TERM with a value known to be
a subset of the current TERM, such as replacing xterm-256color with
xterm.  Otherwise, you should leave it alone.

Trying to do this in profile scripts assumes that you only run local
terminals that come from Fedora and that have been tested.  For example,
if you SSH to a Fedora box from an old xterm that doesn't do 256 colors,
what happens if profile automagically turns xterm into xterm-256color?

-- 
Chris Adams <cmadams(a)hiwaay.net&gt;
Systems and Network Administrator - HiWAAY Internet Services
I don't speak for anybody but myself - that's enough trouble.

[Matthew Garrett]

On Mon, Jun 25, 2012 at 08:47:16PM -0500, Chris Adams wrote:

>  Trying to do this in profile scripts assumes that you only run local
>  terminals that come from Fedora and that have been tested.  For example,
>  if you SSH to a Fedora box from an old xterm that doesn't do 256 colors,
>  what happens if profile automagically turns xterm into xterm-256color? 
The proposal actually handles that by parsing the output of the who 
command, but I'm not sure I'm morally in favour of that.

-- 
Matthew Garrett | mjg59(a)srcf.ucam.org

[Chris Adams]

Once upon a time, Matthew Garrett <mjg59(a)srcf.ucam.org&gt; said:
>  On Mon, Jun 25, 2012 at 08:47:16PM -0500, Chris Adams wrote:
>  > Trying to do this in profile scripts assumes that you only run local
>  > terminals that come from Fedora and that have been tested.  For example,
>  > if you SSH to a Fedora box from an old xterm that doesn't do 256 colors,
>  > what happens if profile automagically turns xterm into xterm-256color?
>  
>  The proposal actually handles that by parsing the output of the who 
>  command, but I'm not sure I'm morally in favour of that. 
That wasn't there when I checked before my email, so I didn't know that.
It sounds like adding one hack on top of another; trying to parse the
output of a command not documented to have a fixed specific format is an
even worse idea IMHO.

The terminal program has very few standard ways to communicate
information to programs running in it:

- the TERM environment variable
- TTY settings (i.e. erase, rows, columns)
- answer-back escape sequences

Trying to use anything outside of that data is a bad idea.  Trying to
divine anything else (or just ignore it and assume you know better) is
bound to fail.

What if another terminal program comes along that only emulates
"traditional" xterm (with only 8 colors)?  How does a profile script
know that everything that says "xterm" can do 256 colors?

Or put it another way: why shouldn't this go in the terminal programs
that support 256 colors?  That way they can be tested, and if any one of
them has a problem, the change can be reverted for just that one (while
the rest that work correctly get the benefit of 256 colors).

I'm also always looking to avoid having more programs automatically run
at the start of a login.  If you've ever had to deal with logging into
an overloaded system, the last thing you want is a profile script doing
"who" and "grep" just to try to override the TERM variable to make it
prettier.  I'd like to see less of that, not more.

-- 
Chris Adams <cmadams(a)hiwaay.net&gt;
Systems and Network Administrator - HiWAAY Internet Services
I don't speak for anybody but myself - that's enough trouble.

[Pádraig Brady]

On 06/26/2012 03:56 AM, Chris Adams wrote:
>  Once upon a time, Matthew Garrett <mjg59(a)srcf.ucam.org&gt; said:
> > On Mon, Jun 25, 2012 at 08:47:16PM -0500, Chris Adams wrote:
> >> Trying to do this in profile scripts assumes that you only run local
> >> terminals that come from Fedora and that have been tested.  For example,
> >> if you SSH to a Fedora box from an old xterm that doesn't do 256 colors,
> >> what happens if profile automagically turns xterm into xterm-256color?
> >
> > The proposal actually handles that by parsing the output of the who 
> > command, but I'm not sure I'm morally in favour of that.
>  
>  That wasn't there when I checked before my email, so I didn't know that.
>  It sounds like adding one hack on top of another; trying to parse the
>  output of a command not documented to have a fixed specific format is an
>  even worse idea IMHO. 
Fair enough.

The main caveat with per terminal settings is that
it might be desired to provide config options per terminal.
Though I suppose users can always override TERM in their
startup files in the unlikely case they need to change
back to 'xterm' for example.

Note the feature is not completed at all.
The presented /etc/profile.d/256colors.sh was only
a 2 minute hack that I thought was worth presenting
as it allows one to easily check the feature and
it does actually work in the vast majority of cases.

>  I'm also always looking to avoid having more programs
> automatically run
>  at the start of a login.  If you've ever had to deal with logging into
>  an overloaded system, the last thing you want is a profile script doing
>  "who" and "grep" just to try to override the TERM variable to make
> it
>  prettier.  I'd like to see less of that, not more. 
Agreed.

cheers,
Pádraig.

[Chris Adams]

Once upon a time, Pádraig Brady <P(a)draigBrady.com&gt; said:
>  The main caveat with per terminal settings is that
>  it might be desired to provide config options per terminal.
>  Though I suppose users can always override TERM in their
>  startup files in the unlikely case they need to change
>  back to 'xterm' for example. 
I still use plain old xterm (the terminal program), and it allows you to
override the TERM variable in a couple of ways.  There's a command-line
option, and there's the old X resource termName.  The newer terminal
programs have configuration menus for various things; do any of them set
it there?  If they don't, I would think it would be relatively easy to
add (and hopefully upstreams would accept such patches).

-- 
Chris Adams <cmadams(a)hiwaay.net&gt;
Systems and Network Administrator - HiWAAY Internet Services
I don't speak for anybody but myself - that's enough trouble.

[Thomas Moschny]

2012/6/26 Chris Adams <cmadams(a)hiwaay.net&gt;:
>  The newer terminal
>  programs have configuration menus for various things; do any of them set
>  it there?  If they don't, I would think it would be relatively easy to
>  add (and hopefully upstreams would accept such patches). 
Tried with XFCE's "Terminal", which has a $TERM option in its
configuration dialog (under advanced options), but it simply doesn't
work. I set it to "xterm-256color", but within the shell, TERM is
still "xterm".

Not sure whether that is an XFCE (or vte) bug, or whether some
system-wide profile script overrides it. Afaict, none of my user
profile scripts overrides it.

- Thomas

[Pádraig Brady]

On 06/26/2012 02:37 PM, Thomas Moschny wrote:
>  2012/6/26 Chris Adams <cmadams(a)hiwaay.net&gt;:
> > The newer terminal
> > programs have configuration menus for various things; do any of them set
> > it there?  If they don't, I would think it would be relatively easy to
> > add (and hopefully upstreams would accept such patches).
>  
>  Tried with XFCE's "Terminal", which has a $TERM option in its
>  configuration dialog (under advanced options), but it simply doesn't
>  work. I set it to "xterm-256color", but within the shell, TERM is
>  still "xterm".
>  
>  Not sure whether that is an XFCE (or vte) bug, or whether some
>  system-wide profile script overrides it. Afaict, none of my user
>  profile scripts overrides it. 
See bottom comment here:
https://bugs.archlinux.org/task/21007#comment94595

[John Ellson]

On 06/26/2012 08:50 AM, Chris Adams wrote:
>  Once upon a time, Pádraig Brady <P(a)draigBrady.com&gt; said:
> > The main caveat with per terminal settings is that
> > it might be desired to provide config options per terminal.
> > Though I suppose users can always override TERM in their
> > startup files in the unlikely case they need to change
> > back to 'xterm' for example.
>  I still use plain old xterm (the terminal program), and it allows you to
>  override the TERM variable in a couple of ways.  There's a command-line
>  option, and there's the old X resource termName.  The newer terminal
>  programs have configuration menus for various things; do any of them set
>  it there?  If they don't, I would think it would be relatively easy to
>  add (and hopefully upstreams would accept such patches).
>

[John Ellson]

On 06/26/2012 06:54 AM, Pádraig Brady wrote:
>  .
>
>  The main caveat with per terminal settings is that
>  it might be desired to provide config options per terminal.
>  Though I suppose users can always override TERM in their
>  startup files in the unlikely case they need to change
>  back to 'xterm' for example.
>  
"Unlikely"  Pffft!!    What else do you resort to every time 
gnome-terminal won't start because dbus has crashed?

John

[Matthew Garrett]

On Tue, Jun 26, 2012 at 10:20:56AM -0400, John Ellson wrote:
>  On 06/26/2012 06:54 AM, Pádraig Brady wrote:
>  >.
>  >
>  >The main caveat with per terminal settings is that
>  >it might be desired to provide config options per terminal.
>  >Though I suppose users can always override TERM in their
>  >startup files in the unlikely case they need to change
>  >back to 'xterm' for example.
>  >
>  
>  "Unlikely"  Pffft!!    What else do you resort to every time
>  gnome-terminal won't start because dbus has crashed? 
Unclear what that has to do with the contents of your TERM variable.

-- 
Matthew Garrett | mjg59(a)srcf.ucam.org

[Adam Jackson]

On Tue, 2012-06-26 at 10:20 -0400, John Ellson wrote:
>  On 06/26/2012 06:54 AM, Pádraig Brady wrote:
>  > The main caveat with per terminal settings is that
>  > it might be desired to provide config options per terminal.
>  > Though I suppose users can always override TERM in their
>  > startup files in the unlikely case they need to change
>  > back to 'xterm' for example.
>  
>  "Unlikely"  Pffft!!    What else do you resort to every time 
>  gnome-terminal won't start because dbus has crashed? 
One never hits that scenario, because:

a) gnome-terminal runs just fine without dbus (unset
DBUS_SESSION_BUS_ADDRESS and run with --disable-factory to verify)

b) when dbus crashes, so does gnome-session, so there's really no
question of launching another terminal

- ajax

[Miloslav Trmač]

On Tue, Jun 26, 2012 at 3:24 AM, Matthew Garrett <mjg59(a)srcf.ucam.org&gt; wrote:
>  We discussed this in fesco today and had a couple of concerns.
Another one is that connecting to systems that don't support xterm-256
is not quite easy.  In particular, there appears to be no way to
configure ~/.ssh/config so that "ssh oldhost" (and "ssh oldhost
arbitrarycommand") transparently changes the TERM value - one would
have to set up shell functions or something similar.  An extension of
the ssh would be very nice - and failing that, an explicit recipe how
to override TERM correctly would be welcome as well.
    Mirek

[Björn Persson]

Miloslav Trmač wrote:
>  Another one is that connecting to systems that don't support
> xterm-256
>  is not quite easy.  In particular, there appears to be no way to
>  configure ~/.ssh/config so that "ssh oldhost" (and "ssh oldhost
>  arbitrarycommand") transparently changes the TERM value - one would
>  have to set up shell functions or something similar.  An extension of
>  the ssh would be very nice - and failing that, an explicit recipe how
>  to override TERM correctly would be welcome as well. 
And I strongly suspect that a correct solution would have to translate 
"xterm-256color" to "xterm", "screen-256color" to
"screen" et cetera. Simply 
setting TERM=xterm in all SSH sessions is probably not sufficient.

Björn Persson

[Chris Adams]

Once upon a time, Miloslav Trmač <mitr(a)volny.cz&gt; said:
>  Another one is that connecting to systems that don't support
> xterm-256
>  is not quite easy.  In particular, there appears to be no way to
>  configure ~/.ssh/config so that "ssh oldhost" (and "ssh oldhost
>  arbitrarycommand") transparently changes the TERM value - one would
>  have to set up shell functions or something similar.  An extension of
>  the ssh would be very nice - and failing that, an explicit recipe how
>  to override TERM correctly would be welcome as well. 
The OpenSSH devs might be open to such.  I'd be willing to take it up
with them and write a patch (if they'll accept it).

Any suggestions on how it should work?  Ideally, some way to say "drop
the -256color suffix from TERM if preset" would be best.

-- 
Chris Adams <cmadams(a)hiwaay.net&gt;
Systems and Network Administrator - HiWAAY Internet Services
I don't speak for anybody but myself - that's enough trouble.

[Pádraig Brady]

On 06/26/2012 07:35 PM, Chris Adams wrote:
>  Once upon a time, Miloslav Trmač <mitr(a)volny.cz&gt; said:
> > Another one is that connecting to systems that don't support xterm-256
> > is not quite easy.  In particular, there appears to be no way to
> > configure ~/.ssh/config so that "ssh oldhost" (and "ssh oldhost
> > arbitrarycommand") transparently changes the TERM value - one would
> > have to set up shell functions or something similar.  An extension of
> > the ssh would be very nice - and failing that, an explicit recipe how
> > to override TERM correctly would be welcome as well.
>  
>  The OpenSSH devs might be open to such.  I'd be willing to take it up
>  with them and write a patch (if they'll accept it).
>  
>  Any suggestions on how it should work?  Ideally, some way to say "drop
>  the -256color suffix from TERM if preset" would be best. 
Well this is a general issue.
For example with urxvt we have:

$ echo $TERM $COLORTERM
rxvt-unicode rxvt-xpm

So sshing from urxvt already needs resetting of $TERM on various systems.

I guess this issue might be some of the reason for the reluctance to address:
https://bugzilla.redhat.com/show_bug.cgi?id=230682

The usual way to set/adjust TERM appropriate for the remote system
is just to use the startup files there. This is what I add to ~/.profile
on a solaris system for example:

[ "$SSH_CONNECTION" ] && export TERM=xterm

I'm not sure adding such functionality to .ssh/config
would be of much benefit TBH.

cheers,
Pádraig.

[Chris Adams]

Once upon a time, Pádraig Brady <P(a)draigBrady.com&gt; said:
>  The usual way to set/adjust TERM appropriate for the remote system
>  is just to use the startup files there. This is what I add to ~/.profile
>  on a solaris system for example: 
Well, that works when the other end is a system that has a shell and
runs login scripts.  I also SSH to routers, switches, firewalls, etc.,
and I need a recognized TERM there as well.  At least some of the
routers I use recognize xterm-256color, but they don't recognize
screen-256color (although I don't know if this proposal adresses
screen, it would be good to have a general solution).

-- 
Chris Adams <cmadams(a)hiwaay.net&gt;
Systems and Network Administrator - HiWAAY Internet Services
I don't speak for anybody but myself - that's enough trouble.

[Kévin Raymond]

Le mardi 26 juin 2012 à 22:31:09 (+0100), Pádraig Brady a écrit : 
>  On 06/26/2012 07:35 PM, Chris Adams wrote:
>  > Once upon a time, Miloslav Trmač <mitr(a)volny.cz&gt; said:
>  >> Another one is that connecting to systems that don't support xterm-256
>  >> is not quite easy.  In particular, there appears to be no way to
>  >> configure ~/.ssh/config so that "ssh oldhost" (and "ssh oldhost
>  >> arbitrarycommand") transparently changes the TERM value - one would
>  >> have to set up shell functions or something similar.  An extension of
>  >> the ssh would be very nice - and failing that, an explicit recipe how
>  >> to override TERM correctly would be welcome as well.
>  > 
>  > The OpenSSH devs might be open to such.  I'd be willing to take it up
>  > with them and write a patch (if they'll accept it). 
Yeah that should be the best thing: not sending $TERM through ssh (even telnet
would need it I think). Needs investigation.


This would avoid many hacks (profile.d, command in authorized_keys…)

[snip]

>  The usual way to set/adjust TERM appropriate for the remote system
>  is just to use the startup files there. This is what I add to ~/.profile
>  on a solaris system for example:
>  
>  [ "$SSH_CONNECTION" ] && export TERM=xterm
>  
>  I'm not sure adding such functionality to .ssh/config
>  would be of much benefit TBH. 
As Chris said, 
the remote needs to be able to run this.


-- 
Kévin Raymond
(Shaiton)

[Michael Stahl]

On 26/06/12 18:45, Miloslav Trmač wrote:
>  On Tue, Jun 26, 2012 at 3:24 AM, Matthew Garrett
> <mjg59(a)srcf.ucam.org&gt; wrote:
> > We discussed this in fesco today and had a couple of concerns.
>  
>  Another one is that connecting to systems that don't support xterm-256
>  is not quite easy.  In particular, there appears to be no way to
>  configure ~/.ssh/config so that "ssh oldhost" (and "ssh oldhost
>  arbitrarycommand") transparently changes the TERM value - one would
>  have to set up shell functions or something similar.  An extension of
>  the ssh would be very nice - and failing that, an explicit recipe how
>  to override TERM correctly would be welcome as well. 
heh, that's exactly why this cruft ended up in my .bashrc  :)

>      # "xterm" does not support color on solaris
>      # https://defect.opensolaris.org/bz/show_bug.cgi?id=1641
>      case $(uname -s) in
>          SunOS)
>              case $TERM in
>                  xterm)
>                      TERM=xtermc
>                      ;;
>              esac
>              ;;
>          # but only Solaris knows "xtermc"
>          *)
>              case $TERM in
>                  xtermc)
>                      TERM=xterm
>                      ;;
>              esac
>              ;;
>      esac 
and that is just to get any colors at all...