Subject: Installation of additional software

From: Guenther Grau <[email-masked]>

Hi,

there has been a thread on the amiga-mailing list of NetBSD
concerning the installation of precompiled ready to
install/deinstall softwarepackages. I tried out the installation
methods of other free operating systems and I talked with many
people about it. What I gathered is, that 

o We need an easy to use install mechanism

o We need an easy to use deinstall mechanism

o We need an easy to use configure tool for sysadmins

What do you think about this? What is the best way to achieve the
high goal of user-friendlyness? For the installation and
especially the deinstallation, it was proposed to create seperate
directories under /usr/local for each packet and sym-link the
bins to /usr/local/bin and the man-pages to /usr/local/man and
all the other files needed. 
This would easy the installation and deinstallation, but
would impose a load of symlinks, which might confuse users, e.g.
when they cd to a symlinked directory and cd .. and find
themselves in a completly different directory. Another proposed
method was to log everything that was installed in a special log
file. When deinstalling a package, the deinstaller could scan the
log file and find out, what files need to be removed. But this
method can be error-prone, too, because if a package is removed
by hand and not using the installer, the log-file would get
inconsistent.

So, what method should be used for the (de)installation of all
the additional precompiled software-packages?

Besides the installation, I think, that we need a decent
sysadmin-tool for sysadmin-beginners. With the advent of
NetBSD-1.0, a lot of casual users might want to install NetBSD on
their home-systems and they don't know nothing about sys-admin.
Therefore we need a decent tool which helps to set up a system in
an easy way (configure networking, adding/removing users,
configure the partitions, ...). What do you think about this?
Yes, I know, this is not the usual BSD-style, but I think that
the user-friendlyness is a very important issue and that we have
to deal with it.

I hope to have started a constructive discussion and not another
flame war, so everyone responding to may email, please don't get
personally and stay to the topic. 

Thanx,

  Guenther

---

From: [email-masked] (Jesus M. Gonzalez)

I saw the message about installing and deinstalling methods,
and I couldn't resist, since we are currently dicssing our local
schema. We are doing everything by hand, and we want to move
to some automated-tools.
> [...]

	We use two directories: /usr/general and /usr/local. The
main point is that we want to consider lots of architectures under
/usr/general, and keep architecture-dependent files unter /usr/local.
We have /usr/general/arch/NetBSD-i386, for example. It is pointed
by /usr/local on NetBSD/i386 machines. Of course, /usr/general is
built from a mixture of NFS and automounter filesystems ---well, it
will be, now we still don't use amd---, so that is accesible under
the same path in all the machines in the network.
 
> themselves in a completly different directory. 

	We keep binaries in separate directories under /usr/local/installed.
We make links from /usr/local/bin, and sometimes from /usr/local/lib.
For man we are currently using MANPATH.
We have found it useful when upgrading to a new version, or when deleting
an application no longer used. It's easy: you just delete (or move)
the directory. The problem comes when trying to locate all the links
that could point to the old directory. But that isn't that hard.
You have to look for oly in a couple of dirs, and sometime you even
dont't have to touch anything: new versions use to name programs like
old ones...

> Another proposed

	The same if installing by hand... I agree, this method can
make life harder when locating things to delete...

> the additional precompiled software-packages?

	We are thinking about a tool that reads a list of files and 
make links to them from /usr/local/bin or whatever. The same
list can be used to deinstall (just by deleting the links and the
dir). If you lost the list, you can do it easily by hand: you have
only well defined places to look at. And at worst, you can get a couple
of dangling symlinks... Using that in combination with /usr/general
(e.g., placing man pages under /usr/general/installed/package_name/man)
can help when you have lots of architectures. 

> the user-friendlyness is a very important issue and that we have

	I also agree with that. I think we need binary repositories for
casual users, and for all of thay don't wanting to bother with compilations
and such... That is one of the best things of Linux, I think. And that
is a place where we, the non-kernel-hackers can help.

	That's all,

		Jesus.

-- 
Jesus M. Gonzalez Barahona    | Universidad Carlos III (Madrid, Spain)
tel: +34 1 624 94 58          | e-mail: [email-masked]
fax: +34 1 624 94 30          |         [email-masked]
(Sometimes our headers are not o.k., please reply to any of this addresses)
.From within Universidad Carlos III, you can better use [email-masked]

---

From: Hubert Feyrer <[email-masked]>

> For man we are currently using MANPATH.

I guess linking to /usr/local/man would be ok, too. :)

> can help when you have lots of architectures.

I've written such an install-script. You tell it which binaries,
man-pages, etc. to install, and it will link them from the
"package-private" directory (/usr/general, or whatever) into a
"public" directory (/usr/local/bin, .../man, ...).

If you're interrested in the script (there's alsoa uninstall-script),
you can get it on ftp.uni-regensburg.de (the inofficial main site for
NetBSD/amiga) in /pub/NetBSD-Amiga/contrib/EasyInstall-1.0.tar.gz. To
read the README, type: "sed s,/opt,/usr/local,g README | more" (As I
thought of using another directory for placing packages, I adopted
SVR4's /opt system, and got heavy flames for calling it so... maybe
/usr/general is just better? ;-)

> is a place where we, the non-kernel-hackers can help.

Sure as hell true. We've even started putting up such a reprository
for NetBSD/amiga on ftp.uni-regensburg.de, but right now it's rather a
throw-together of some tar-archives, with different packages using
different install-methods, -directories, etc.

Besides that, I've got a perl-based adduser/deluser-script, which
could be used to simplify adding users. If someone is interrested in
it, leave me a note and I'll put it up.

Hubert

=============== Hubert Feyrer ============================================
      Weekdays: Rennerstr. 19, D-93053 Regensburg,  Tel. 0941/701788
      Weekends: Bachstr. 40,   D-84066 Mallersdorf, Tel. 08772/6084
      Internet: [email-masked] == IRC: hubertf
==========================================================================

---

From: [email-masked] (Jesus M. Gonzalez)

> I guess linking to /usr/local/man would be ok, too. :)

	We usually prefer using MANPATH. Directories under man tend
to be large, and its difficult to know what man pages are part
fo what packages... MANPATH deals with it in a simple and flexible
way. And when you want to delete a package, you just rm the directory
under installed... Of course this may be diffeent if you have a tool...
Then you could just use it to delete the man pages... if you have
a file with the info.

> "public" directory (/usr/local/bin, .../man, ...).

	Well, the thing is a bit more complicated.... Under /usr/general/arch
you have the 'real dirs' for different OSs. /usr/local/bin in NetBSD is,
for example, a link to /usr/general/arch/netbsd/usr.local/bin.
When you want to get a package and install it, you first go to
/usr/general/distrib (if distirubtion is source) or to
/usr/local/packages/distrib (if its binary for the architecture).
You open a new dir there, and ftp the package. Then, if it is
'ready to run' (eg., binaries, Tcl/Tk scripts and so on), you
open a dir under /usr/local/packages/installed or under
/usr/general/installed, and unpack the thing there (usually, with
bin, doc, lib, man subdirectories). The you config things so that
e.g. config files are found, and make links *only for the binaries*
(and libs, if other packages may need them). Add the new man path
to MANPATH, and everithing is ready. All machines of that architecture
on the net (or *all* the machines, if the thing was general)
see the package, and can work with it.

	If the package needs to be compiled, I use to unpack it
under /usr/local/packages/test, configure and compile there, test it
if possible, and install under /usr/local/packages/installed.

	Anyway, most of this scheme is 'static', in the sense that
you need only having a couple of links in all the machines
(for example, from /usr/local to /usr/general/arch/OS-type/usr.local),
and /usr/general importe by all the machines. The only `dynamic'
links ---that must be done per package--- are those under 
/usr/local/bin... That where your tool can be very useful.

> /usr/general is just better? ;-)

	Well, I'll try your tool... Sounds good. (about /opt, yes
I prefer /usr/general, but that may be a religious war.... ;-) )
> different install-methods, -directories, etc.

	Maybe this is the time for discussing this... I have some
space in a ftp server, wich I can devote to NetBSD/i386. Maybe
now, with the upcoming 1.0, it's the right moment...

	What do you think, netbsd users?

		Jesus.

---

From: John Kohl <[email-masked]>

I'm going to look at the FreeBSD pkgadd stuff sometime soon; if it does
reasonable things I see no reason to reinvent this wheel.

==John

---

From: Mark Gooderum <[email-masked]>

BTW...

I can't devote much time to the pkgadd part, but I'd be willing to do
the bridge makefile stuff and maintain the sets of needed patches for
different packages (since it will always take an author time to incorperate
changes...in the interem, a patch source will be needed).

Just quieting the "very nice, do it" arguments...

My machine at home is up on the Internet full time...so I can keep some
stuff available and handle mailing lists or whatever (majordomo is up), but
I don't have a lot of disk and I'm on the back end of a ~28.8k dedicated
modem link, so handling tons of megs of the packages themselves probably
isn't viable for now...

-Mark Gooderum
 [email-masked]

---

From: Mark Gooderum <[email-masked]>

Some quick comments....

First, people seem to be mixing up two issues...one is how to bundle
and install software, particularly in such a way that it can be easily
installed, noted, and deinstalled.

Second is how to manage filesystem trees and program locations
in a multiple architechture/OS etc environment.

They really are two seperate issues...

I think the first issue is the one that should be addressed, and it should
be done in such a way that is flexible enough that Joe Systemdude can 
do #2.  This is because every site that has the #2 problem has solved it
differently, and they won't change the way they do it just to be able
to use XYZ's operating system install package neatly.

I came from an envionment where we ran a fairly uniform environment accross
seven different Unixes.  Suffice to say our filesystem tree setup was
pretty complex, although not nearly as much as larger sites I've visited.

The point of all this was that we couldn't really change the way we did 
things just to make "pkgadd" happy for instance.

In general sites tend to install either one package per dir/tree (ala
/usr/local/wp51, /usr/local/frame, etc) or everything in one tree (ala
/usr/local), epecially free software.  The most common reason I here for
seperate dirs is either A.) the product install won't support anything
else, common for commercial software, or B.) they want an easy way to 
cleanup or identify a package.

The pkg*** stuff solves B nicely.  I haven't used FreeBSD's, but Solaris'
is pretty nice.  You can install, uninstall, backout, patch, etc and know
what's been done.  Shared files get a reference count, and there is
even dependancy tracking.  The major down sides are that the key information
is kept in one *huge* text file, which is extremely subject to corruption,
or becoming out of date, and is slow to update.  For instance, the package
tools will barf and refuse to update/uninstall or otherwise
manipulate a package if the contents don't jive with the history perfectly.
The other problem is it doesn't deal well with server/client packages where
you may have a large package that has several common files that go on the
server, but may have a few config files that change on the kernel.

I think the key requirements of any binary package system are:

	install is easy, and flexible, and verifable
		-Ie: support for installing into any tree, seperate or
		 shared, with maybe an (optional) arch/arch independant
		 split
	tracking is robust and prevents corruption, or at least can punt
		if things get messed up
	uninstall is easy
	installed packages can be identified
	a file can be easily identified with a given package

Other pluses that are nice icing but I don't think are essential:

	post-install/post-deinstall scripts for needed configuration/
		customization (or unXXX)
	verification - compare a package bundle against an installed package,
		useful for security and recovery reasons (which files did 
		I loose on that disk crash or site breakin), also for which
		files did I need to change for customization
	reconstruction of package tracking info...given the install media/
		package and the same input to the install package program, 
		it should be possible to reconstruct the tracking info of
		an already installed package
	server versus client side packages (maybe with an update to a server
		side config that tracks which clients depend on the server
		package)
	file tracking supports the concept of immutable files versus those
		that may change (so for instance it will complain bitterly
		if a binary changes, but maybe warns and offers to make
		a backup if a config file changes and is being overwritten)
	patching support - ability to update add files and maybe save orignals
	shared file reference count - ability to share files between packages
	package dependancy tracking (needs to include versioning to be 
		usable)
	versioning of packages
	arch versus arch independant parts support
	moving - move an installed package to a new location iwth local 
		customization intact and update tracking (much easier than
		deinstall/reinstall/recustomize)

The Solaris pkgadd package does most of these except moving, tracking
info robustness, server/client tracking), and immutable/changeable files.

I don't know much about the FreeBSD pkg stuff...

Note that all of this is from the point of view of the user for a binary
package.  There are issues for developer's tools for creating a package.

For sources I'm very much of the school of get changes incorperated for
native support of NetBSD in the package.  I'm also opposed to totureing
the build process into the BSD Makefile format, this is especially a big
loss for autoconfig based packages.

What would be useful is a set of BSD -> Native source "bridge" makefiles.
These would exist in the parent dir of the package (to avoid makefile
name collision).  The parent makefile would include a sub-dir specific 
makefile that would then invoke the right make (many packages, like many
Imake makefiles and some autoconf built makefiles won't work with BSD make,
they need GNU make) or other command on the appropriate target in the
subdir.

The three "templates" for this I can think of for this would be a minimal
common makefile (that just passed a make, make install, make clean type set
of make actions to the subdir makefile).  The other two would be for autoconf
based packages and for Imake based packages.  The second two would be more
complex and include a pre-make target that would run xmkmf or configure 
appropriately (getting the needed arguments from a common config, so that
you could globally specifiy a --prefix and --exec-prefix for instance), and
other bits (like a make distclean, re-configure if you're building a 
different arch).  Note that the autoconf based version could support multiple
arches w/o reconfiguring using arch specific obj dirs (sound familiar?) and
the --srcdir directive to autoconf.

Well, nuff said for now,

Mark

---

From: Andrew Cagney <[email-masked]>

Excerpts from mail: 14-Sep-94 Re: Installation of additio.. Mark
[email-masked] (5984)

> cleanup or identify a package.

Add: C) Have concurrent versions installed (testing/compatibility); D)
back out of a package update instantly.

				regards Andrew