GOTCHA in Ubuntu - broken shell

[Bill Sconce]

Hi, all -

You may know this: Ubuntu's default shell isn't bash.

[I grepped to see if anyone has mentioned the issue here
before; it seems not. Apology if my post is a duplicate.]

For months I've been seeing difficult-to-understand problems in
shell scripts: syntax errors, "bad substitution"; weird stuff. 
Not reproducible. Dozens of hours lost, trying workarounds. 
I am not making this up.

Did you know that Ubuntu's /bin/sh, which used to point to bash
(as it does and always has on most Linux distributions), was
changed to "dash"?  (Subsequent to LTS/6.06, evidently.)

True, dash mostly works. Where it doesn't work...  Arrgh.  

I have countless scripts which begin with "#!/bin/sh". Moreover,
when I've taught "Introduction To Scripting" I've always suggested 
that scripts begin with "#!/bin/sh", since that matches what
students will see in any distro's installed scripts. (I know, 
I know, I've ALWAYS known that /bin/sh *could* be other than bash.)
"#!/bin/sh" has been "accepted practice" on Linux. Always. Even in
Ubuntu.

BUT. 

Ubuntu made a change. Many users therefore may have scripts which
had worked fine since the days of Yggdrasil (as in my case), and
which indeed worked fine through several releases of Ubuntu, but
which for the last year have been causing transient, hard-to-find,
"WTF?" problems.

Solution: not difficult. Make your shebang line be "#!/bin/bash".

(And remember to not trust Ubuntu. They don't think things through
to the consequences. They don't listen, either. See below.)

Hope this helps someone.

-Bill

_______________________________________
Details; the light finally dawned just this morning.  A script
fragment which had worked for years (interactively, in terminal
windows) seemed to suddenly break when used in a script. It
involved slicing. Interactively (in bash):
  $ breakfast="spamandeggs"
  $ echo "${breakfast:0:4}"
  spam
  $ 
In dash (i.e., in a script which starts with #!/bin/sh on Ubuntu):
  $ breakfast="spamandeggs"
  $ echo "${breakfast:0:4}"
  /bin/sh: Syntax error: Bad substitution
  $ 
The message is correct, of course: it's the shell which doesn't like
my code, and it's /bin/sh which is the shell. That's the tipoff.
>From there,
  $ ls -al /bin/sh
  lrwxrwxrwx 1 root root 4 2007-01-01 17:30 /bin/sh -> dash
  $ 
  
** NOT bash! **

It had never occurred to me THAT A DISTRO COULD DO SOMETHING AS 
DANGEROUSLY REGRESSIVE as change the default shell to something
not backward compatible.

_______________________________________
Another example. This one is [was] my "favorite"; I think it was
the first one to bork on me. The error message was [is] so obscure
that for the past year I've lived with band-aiding each "if" 
statement one at a time, just to have something which works.
(One uses "if" statements much more frequently than slicing.)
In bash:
  $ if [ "${breakfast}" == "spamandeggs" ]; then
  >   echo "yummy"
  > fi
  yummy
  $ 
In dash:
  $ if [ "${breakfast}" == "spamandeggs" ]; then
  >   echo "yummy"
  > fi
  [: 11: ==: unexpected operator
  $ 
  
How ugly and unhelpful is that?  

The band-aid, by the way, was that "=" instead of "==" works.
(And how ugly is THAT? And how do you explain that to a student?)

____________________________________________
Some background.

"Awww, c'mon. It couldn't be the *shell*."
"They couldn't have been that stupid."

They could.

"Awww. I couldn't have been stupid enough to believe that
it was me, for over a year."

I could...

https://bugs.launchpad.net/ubuntu/+source/dash/+bug/61463
  Bug #61463, first reported on 2006-09-20  by  Egil Hasting
  Bug description
    The scripting capability between bash and dash are not
    equal, which could break scripts that today relies on bash
    ( #!/bin/sh -> /bin/bash), i tracked down this problem
    with cedega on edgy, and was able to get it working again
    by forcing the script to use #!/bin/bash (instead of /bin/sh)
  ______________________________
  Tom Bruno wrote on 2006-09-27:
    This is a pretty large scale problem. I'm seeing scripts breaking
  all over the place. and strange errors about bad interpreters and
  such since this change.
  ______________________________
  Jason Straight (LeeJunFan) wrote on 2006-09-29:
    Yeah, I agree, I even had some installation scripts fail when
    dash replaced bash during apt-get dist-upgrade. As well as some
    LSB scripts. It was an unholy mess.
  ______________________________
  anthony baxter wrote on 2006-10-17:
      Please reverse this change before edgy final. It's caused 
    massive breakage for me - for instance, the intel compiler was
    utterly broken. It relied on 'export -n' and 'exec -a' working.
    I'm almost tempted to remove dash with dpkg -r and live with
    the apt-get complaints.
      This is a huge mistake - it's broken all manner of things
    for me, and I suspect that it will end up being a support
    nightmare once edgy final is released to the world.
      Yes, people shouldn't have assumed that /bin/sh was bash 
    rather than bourne shell - but they have. I can fix my own
    scripts, but going through other vendor supplied scripts is
    going to be a complete pain.
  ______________________________
  Graham Hawkins wrote on 2006-10-21:
      Small beer compared with the problems above, but it also
      breaks Limewire's runLime.sh
  ______________________________
  sparr wrote on 2006-10-28:
      Complain to the script authors who are making the mistake.
    Dash has been a viable provider of /bin/sh (by way of debconf
    on the dash package) for a long time. There are other shells
    that provide 'sh' functionality too. If a script really 
    requires /bin/bash then it needs to start with #!/bin/bash.
      Analogy: If I needed a square, I would not ask you for a
    rectangle. Although you might give me the square that I need,
    since that would fulfill my stated rectangle requirements, it
    is not guaranteed.
      People who need /bin/bash should not ask for /bin/sh and hope
    that the result they get just happens to be /bin/bash. I have
    used dash as my /bin/sh for over a year now. It is insanely faster,
    especially for huge shell scripts such as 'configure' in large
    projects.
  ______________________________
  Stephen Thorne wrote on 2006-10-31:
      Suggested resolution: Use /bin/dash instead of /bin/sh for
    scripts that are desired to run fast, and revert the change.
      If you like ./configure running faster, then patch the code
    so that ./configure has #!/bin/dash.
      This change has obviously caused regressions, and should be
      considered a high priority bug that should be fixed, not
      justified.
  ______________________________
  sparr  wrote on 2006-10-31:
      The 'bug' is not with dash, it is with every package that
    dash "breaks". They should all be fixed. *maybe* dash should
    not be the default until they are fixed, but I think they
    would never get fixed if it wasn't.
  ______________________________
  LionsPhil wrote on 2006-11-16:
      Sparr, your comments are unhelpful---there IS a problem here,
    and it making the distribution notably less useful. Reverting
    /bin/sh to point to bash will fix this until you can (quite
    rightly) beat people into specifying /bin/bash if they need bash.
      But, for now, this is breaking stuff, and it can be easily
    unbroken. The high road here is not helpful to Ubuntu users,
    and smacks of the same kind of perfectionist arrogance which
    curses open source software to forever be a continual battle
    with your computer to make it actually work properly.
      If you think this is minor, search for "dash" within the 
    Ubuntu bug list. I've seen this break _configure scripts_, 
    for crying out loud. You're not going to upstream fix
    autoconf _and_ get everyone who uses it to release new tarballs
    this side of 2007.
  ______________________________
  Simon Howard wrote on 2006-11-16:
      I would have thought it would be preferable to have a system
    that works than a fast one that doesn't. Besides, is bash really
    that slow?
      Yes, ideally shell scripts that use #!/bin/sh shouldn't rely
    on bash features, but the truth is that a lot of them do, because
    every other distribution out there uses bash for sh. You can tell
    people to complain to the script authors, but there is going to
    be another script that doesn't work. And another. And another.
    And a thousand more.
      What I find perplexing is that Ubuntu of all distributions would
    make a decision like this, when from the very beginning it has
    always seemed to have had the goal of being easy to use and of
    "Just working". This is the kind of subtle and insidious
    difference that nobody is going to notice without hours of
    searching and experimentation. In the end people are going to
    just decide that "Ubuntu is broken" and move on.
      By the way, the installer for Borland Starteam also doesn't
    work. Add that to the list of software that Ubuntu now no longer
    supports. At work my team is looking to standardise on Ubuntu as
    our Linux development OS, but if we can't use this then I expect
    we'll end up using Fedora or another system. I'll be very
    disappointed if this happens.
      Good engineering involves making compromises. I hope you'll
    come to the right decision.
    
            [...and so on, for dozens of comments through
             the following year, up to the most recent post,
             two weeks ago...  -Bill]
  ______________________________
  chemist109 wrote on 2007-09-08:
    I don't get it. If the devs want faster script execution why
  not just make the she-bang #!/bin/dash for their scripts? Dash
  isn't _sh_ any more than bash is. And, dash isn't 100% sh 
  compatible either, is it? I don't know, it seems ridiculous to
  me to make this change and then just say "suck it up" when
  numerous things break.