shell, perl, performance, parallelism, profiling, etc.

[Jerry Feldman]

On 10/22/2008 11:34 AM, Ben Scott wrote:
> I ran did several test runs in succession. The goal is to get
> everything cached in RAM for these trials.  (Artificial, to be sure,
> but it's harder to normalize the effects of disk I/O.)  The first run
> in a series was typically different.  I threw those results out.  The
> results I have been reporting are the typical case, after everything
> is cached.  It seems to be stable after that first run.  (Within +/-
> 0.1 seconds "real".)
>
>   
>> Additionally, Perl compiles the script first.
>>     
>
>   Hmmm, that's an interesting point.  Still, once compiled, Perl
> scripts are supposed to run faster.  bash has to parse and interpret
> everything as it goes.  In this case, the shell variant doesn't have
> any looping, so I would expect it to be a wash.
>
>   If there was any kind of loop in the shell script, I would expect
> the Perl variant to be much faster at the task.
>
>   Hmmm, I suppose another experiment would be to turn the shell
> variant into a Perl script, without using any Perl constructs beyond
> what the shell variant does.  I'll try that next.
>
>   
>> Another was that you can look at performance is to use a C shell script on
>> a system where the C Shell is not used as a login shell, so it is
>> generally not resident.
>>     
>
>   csh/tcsh would be resident after the first run.  :-)
>   
Yes, the csh/tcsh would be resident after the first run, but so would 
perl. Probably a reasonable benchmark would be to run the scripts 
several times in successtion, and throw out the first run. This would 
most likely compensate for the load time of the Perl compiler/interpreter.

Another thing is comparing a compiled program to an interpreted program. 
I have seem some cases where interpreted programs can be made to run 
faster than some compiled programs. I have actually seen that with 
FORTH, but the traditional wisdom that compiled programs generally run 
faster than interpreted programs holds true most of the time.

However, my thought is that if you are concerned with the performance of 
a shell or Perl script, then you have the wrong language, and should 
consider a more traditional compiled language. On the BLU server, we had 
a script that would convert mailman passwords to htpasswords. The shell 
script took many minuted (I think it was over 30 at the time) where I 
rewrote it in C++, and it took something like seconds. Scripts, whether 
written in BASH, CSH, or Perl are very useful, but they have their 
place. I've seen whole applications written entirely in scripts 
(Unixshell, Perl, DCL), but once performance becomes an issue then you 
really need a binary solution.

-- 
Jerry Feldman <gaf at blu.org>
Boston Linux and Unix
PGP key id: 537C5846
PGP Key fingerprint: 3D1B 8377 A3C0 A5F2 ECBB  CA3B 4607 4319 537C 5846


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 251 bytes
Desc: OpenPGP digital signature
Url : http://mail.gnhlug.org/mailman/private/gnhlug-discuss/attachments/20081022/a935c65e/attachment.bin