LXF Magazine DVD Index

[Jim Kuzdrall]

Greetings,

    The bash script which follows builds an index of all source code 
files shipped on the DVDs that come with a subscription to Linux Format 
magazine.  I searched in vain for one on the Internet without luck.  I 
hope I didn't overlook something better.

    The DVDs are a great solution for those who can only get (or choose)
56K dialup service.  It delivers new copies of many Linux distributions 
(often bootable from the DVD), kernel source code, desktop source code, 
and applications which can avoid 50MB to 300MB downloads.

    The index answers the question: Was a recent copy of the software I 
suddenly crave on some previous DVD?  If so, I won't have to download a 
version to try it.

    The first bash script, lxf_add.sh, compiles the index.  Call it with 
no parameters or -h or --help for instructions (see code).

    The second script, lxf_find.sh, does the search.  Call it with no 
parameters for instructions (see code).  It should accept regular 
expressions properly (I think).

    I have built an index from DVD 47 to 103, which we might get onto 
the Wiki if anyone is interested (330K zipped).

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ lxf_add.sh
#! /bin/bash

# Add source files from the latest Linux Format magazine DVD to a
# composite index for searching.  Use with  lxf_find.sh to locate
# the latest version of a program available on the DVD set.

# Bash script: lxf_add.sh
# Author: James A. Kuzdrall  inq at intrel.com
# License: GNU General Public License (GPL)
# History:
# 11-Feb-08 jak; created

# if no DVD name given, show help
if [ "$#" -eq 0 -o "$1" = "--help" -o "$1" = "-h" ]; then
cat << EOT

-----------------------------------------------------------------------
Command line format:
  $0 <LXFDVD###>
    where ### is the number of the mounted Linux Format magazine DVD

Adds the latest Linux Format DVD source file names to a composite index.
  Index files lxfdvd_inx.csv and lxfdvd_last.csv.
  Field separator is white-space (in particular, a space).
  The fields are:  1) 'source_name.[bz2|deb|gz|iso|rpm|tgz]'
                   2) ' DVD_identifier'.

Examples of use:
  Add all source files in the DVD named LXFDVD102...
      lxf_add.sh LXFDVD102
  To find DVDs which contain a version of OpenOffice...
      lxf_find.sh openoffice
----------------------------------------------------------------------

EOT
exit 1
fi

# Please set these paths appropriately for your computer
# mnt_path=""
# sum_path=""
mnt_path=/media/
sum_path=/home/jak/aaajak/lnx/src/aaa_lxf/

# check that user initialized paths
if [ "$mnt_path" = "" ] ; then
  echo
  echo -------------------------
  echo "  In $0, please initialize mnt_path, where DVD is mounted"
  echo "    (usually /mnt/ or /media/)"
  if [ ! "$sum_path" = "" ]; then
    echo -------------------------
    echo
    exit 2
  fi
fi

if [ "$sum_path" = "" ] ; then
  if [ ! "$mnt_path" = "" ] ; then
    echo
    echo -------------------------
  fi
  echo "  In $0, please initialize sum_path, where summary files go"
  echo '    (Perhaps create "lxf_dvds/" in your home directory)'
  echo -------------------------
  echo
  exit 3
fi

# program-wide definitions:
dvd_nam=$1
mnt_dir=$mnt_path$dvd_nam/
ix_file=$mnt_dir".index/index.xml.gz"
ix_file_alt=$mnt_dir".index/index.xml"
sum_file=$sum_path"lxfdvd_inx.csv"
sum_last=$sum_path"lxfdvd_last.csv"
sum_bkup=$sum_path"lxf_"$dvd_nam"_inx.csv"
sum_tmp=$sum_path"tmp.csv"

# see if the DVD is mounted
if [ ! -d $mnt_dir ]; then
  echo --------------------------
  echo "  DVD $dvd_nam does not appear to be mounted at $mnt_dir"
  echo --------------------------
  exit 4
fi

# see if index file is there
if [ ! -e $ix_file -a ! -e $ix_file_alt ] ; then
  echo --------------------------
  echo "  $ix_file not found on this DVD"
  echo --------------------------
  exit 5
fi

# if DVD index file is not zipped, create a zipped version
if [ -e $ix_file_alt ] ; then
  cp $ix_file_alt $sum_path"index.xml"
  ix_file=$sum_path"index.xml.gz"
  gzip $sum_path"index.xml"
fi

# extract the index; convert all tabs to spaces
# pass thru: from line begins "<package name=" to line ends "</file>"
# keep lines ending in "/disk>" or beginning with "<file name"
# eliminate all but those with source ext or "path"
# strip xml tags from "disk" line, saving DVD name with @ appended
#     in holding buffer
# strip xml tags from "file name" line, inserting saved DVD name after
# change all \n to spaces, making one line; divide by making @ -> \n
# eliminate leading spaces; save in temp file (can't sort from pipe)
zcat $ix_file | tr '\t' ' ' |
  sed -n -e '/^ *<package name=.*/,/^ {1,}<\/file>$/p' |
  sed -n -e '/<\/disk>$/p' -e '/^ *<file name=/p' |
  egrep '(disk|deb|rpm|bz2|tgz|gz|iso)"*>$' |
  sed -e '/^ *<disk/{ s/ *<disk>//; s/<\/disk>/@/; h; d }' \
      -e 's/<file name="//' -e 's/">$//'  -e 'G' |
  tr '\n' ' ' | tr '@' '\n' |
  sed -e 's/^  *//' > $sum_tmp

# sort; lines begin with source file name
sort --ignore-case < $sum_tmp > $sum_bkup

# append the new single-DVD entries to all previous entries
cat $sum_bkup >> $sum_file

# eliminate duplicates in case same DVD appended twice
sort --ignore-case < $sum_file > $sum_tmp
uniq --ignore-case < $sum_tmp > $sum_file

# Write a file with just the latest version available
sort --ignore-case --reverse < $sum_file > $sum_last
sort --ignore-case --key=1,1 --unique < $sum_last > $sum_tmp
sort < $sum_tmp > $sum_last

# remove the temporary and the compressed files, if they exist
rm -f $sum_tmp $sum_path"index.xml.gz"

exit 0

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ lxf_find.sh
#! /bin/bash

# lxf_find.sh
# Find source files in the Linux Format DVD composite index
# Index file, lxfdvd_inx.csv, is in white-space separated value format.
# The fields are: 1) 'name.[rpm|bz2|tgz|gz]' 2) ' DVD_identifier'.

# Bash script: lxf_find.sh
# Author: James A. Kuzdrall  inq at intrel.com
# License: GNU General Public License (GPL)
# History:
# 11-Feb-08 jak; created


# if no parameters or "help" requested, show instructions
if [ "$#" -eq 0 -o "$1" = "--help" -o "$1" = "-h" ]; then
cat << EOT

---------------------------------------------------------------------
Find source files that were published on the Linux Format magazine DVD.
  lxf_find [-a|--all] search-name
  (the search is case-independent)

Examples of use:
  Find latest DVD files with names containing ghost...
        lxf_find.sh ghost
  Use regular expression to limit files to those beginning with ghost...
        lxf_find.sh '^ghost'

Note -a --all:
  Identical source files are often repeated on several DVDs, e.g. 
ncurses.
    lxf_find normally lists only the latest DVD.
    But, sometimes that DVD may be missing or unreadable.
    Use --all to include the names of all DVDs containing the file.
-----------------------------------------------------------------------

EOT
exit 1
fi

# local definitions (change to suit your system):
# sum_path=""
sum_path=/home/jak/aaajak/lnx/src/aaa_lxf/


if [ "$sum_path" = "" ] ; then
  echo -------------------------
  echo "  In $0, please initialize sum_path"
  echo "    That is where summary files created by lxf_sum.sh reside"
  echo '    (Perhaps it is "lxf_dvds/" in your home directory)'
  echo -------------------------
  echo
  exit 2
fi

# standard definitions from lxf_add.sh
sum_file=$sum_path"lxfdvd_inx.csv"
sum_last=$sum_path"lxfdvd_last.csv"

# list all DVDs that have the file?
if [ "$1" = "-a" -o "$1" = "--all" ] ; then
  search_in=$sum_file
  search_phrase=$2
else
  search_in=$sum_last
  search_phrase=$1
fi

# does the file exist
if [ ! -e $search_in ] ; then
  echo Missing: LXF DVD summary file $search_in
  exit 3
fi

# extract the matching files from DVD index
echo ---------------------------------------
grep $search_phrase $search_in
echo ---------------------------------------

exit 0