Moving and Upgrading Bugzilla

May 27, 2018

I had to migrate and upgrade a Bugzilla installation to a newer machine with Ubuntu LTS installed. Fortunately, Bugzilla has documented the steps in Moving Bugzilla Between Machines, but unfortunately the documentation does not seem to be uptodate.

First I had to make sure the latest update to the LTS was installed:

sudo apt update
sudo apt upgrade
sudo apt dist-upgrade

This resulted in

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.4 LTS
Release: 16.04
Codename: xenial

Next, I following the Quick Start installation guide

$ sudo apt-get install git
[sudo] password for herbert:
Reading package lists... Done
Building dependency tree
Reading state information... Done
git is already the newest version (1:2.7.4-0ubuntu1.3).

The next recommended step brought up the error message

Package apache2-mpm-prefork is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

E: Package ‘apache2-mpm-prefork’ has no installation candidate

and I had to remove the package apache2-mpm-prefork from the command

sudo apt-get install apache2 mysql-server libappconfig-perl libdate-calc-perl libtemplate-perl libmime-perl build-essential libdatetime-timezone-perl libdatetime-perl libemail-sender-perl libemail-mime-perl libemail-mime-modifier-perl libdbi-perl libdbd-mysql-perl libcgi-pm-perl libmath-random-isaac-perl libmath-random-isaac-xs-perl libapache2-mod-perl2 libapache2-mod-perl2-dev libchart-perl libxml-perl libxml-twig-perl perlmagick libgd-graph-perl libtemplate-plugin-gd-perl libsoap-lite-perl libhtml-scrubber-perl libjson-rpc-perl libdaemon-generic-perl libtheschwartz-perl libtest-taint-perl libauthen-radius-perl libfile-slurp-perl libencode-detect-perl libmodule-build-perl libnet-ldap-perl libauthen-sasl-perl libtemplate-perl-doc libfile-mimeinfo-perl libhtml-formattext-withlinks-perl libfile-which-perl libgd-dev libmysqlclient-dev lynx-cur graphviz python-sphinx rst2pdf

I cloned Bugzilla into /var/www using git

/var/www$ sudo git clone --branch release-5.0-stable https://github.com/bugzilla/bugzilla bugzilla

and edited the MySql config files as recommended

/etc/mysql/mysql.conf.d$ sudo nano mysqld.cnf

max_allowed_packet = 100M
ft_min_word_len = 2

Create a bugs user account and run mysql "GRANT ALL PRIVILEGES ...." to create the bugs database

Next, I created a backup of the original Bugzilla database, and proceded according to the Moving… documentation.

/var/www/bugzilla$ sudo ./checksetup.pl

Check setup brought up warnings about missing Perl modules.

/var/www/bugzilla$ sudo /usr/bin/perl install-module.pl --all
ERROR: Using install-module.pl requires that you install "make".

Solve this error by installing make:

$ sudo apt-get install make

The next run of Check Setup raised another error message:

/var/www/bugzilla$ sudo ./checksetup.pl

DBD::mysql::db do failed: Cannot change column 'setter_id': used in a foreign key constraint 'fk_flags_setter_id_profiles_userid' [for Statement "ALTER TABLE flags CHANGE COLUMN
setter_id setter_id mediumint NOT NULL"] at Bugzilla/DB.pm line 742.
Bugzilla::DB::bz_alter_column_raw(Bugzilla::DB::Mysql=HASH(0xc89c6dc), "flags", "setter_id", HASH(0xca65fc0), HASH(0xe69a248), undef) called at Bugzilla/DB.pm line 701
Bugzilla::DB::bz_alter_column(Bugzilla::DB::Mysql=HASH(0xc89c6dc), "flags", "setter_id", HASH(0xca65fc0)) called at Bugzilla/Install/DB.pm line 626
Bugzilla::Install::DB::update_table_definitions(HASH(0x97e6884)) called at ./checksetup.pl line 172

I found a couple of forums regarding this error (Mozilla Support, SO), and the problems seems to be caused by newer MySql versions refusing to change or delete a column which is referenced by a foreign key constraint.
I removed the constraint directly from the mysql prompt:

use bugs;
alter table `flags` drop foreign key `fk_flags_setter_id_profiles_userid` ;

After this, the Check Setup script completed without errors.

 


Recovering a Corrupted Mailman Installation

December 16, 2013

One day a Mailman installation that I maintain became corrupted.

Instead of presenting the typical web interface, it only showed the directory listing or returned 404’s, just as if the Python handler could not be found to process the requests.

Anxious to lose the configuration and content of several mail lists, I followed two tutorials to backup the Mailman data:

After completing the backups, I gave it a try and re-installed Mailman. However, the first uninstall statement

apt-get remove mailman

stopped with a message that some directories still contained data, so I had to manually delete the “bounces” directory under /var/lib/mailman/qfiles.

Finally, the statements

apt-get remove mailman
apt-get install mailman

were executed successfully, and Mailman was up and running again (without restoring any of the backups).


Recovering Ubuntu 8.04 LTS from Failed Harddisk

April 30, 2009

You may have noticed that the web server hosting my homepage and download site has been down since the weekend. I noticed problems when the sites responded to request with MySQL errors.

Running fsck returned the message “Bad magic number in super-block” which means that e2fsck cannot completely repair the disk. It turned out that the partition table had been destroyed, but mke2fs -n still found some superblocks.

Started up the PC with a Knoppix Live CD to repair the broken disk.

In couple of forums I found the utility TestDisk which re-creates lost partitions, and it both recovered the boot and the swap partition. However, e2fsck still failed with the messages:

Attempt to read block from filesystem resulted in short read
Attempt to read block from filesystem resulted in short read
reading journal superblock
Attempt to read block from filesystem resulted in short read
while checking ext3 journal

It was now clear that the harddisk could not be repaired, so I got a new one and copied the original harddisk with a program called Ddrescue.

Ddrescue is a great tool (documentation), as it copies one device onto another, displaying the number of read errors and the size of the erroneous blocks. The amazing thing (if you don’t know how it works) is that the error size *reduces* after the first full scan of the source disk. The initial 2.5GB of unreadable disk finally reduced to about 15MB.

After ddrescue was finished, I ran another fsck on the new disk, this time successfully. Time to reboot.

Reboot brought a black screen, with “1234F” the only thing displayed. It turned out that that was the remainder of the TestDisk MBR which could not find a bootable partition. Need to get GRUB back.

The Knoppix disk would not help me now (disks are named /dev/hda instead of /dev/sda), but fortunately I had a Ubuntu 8.10 Desktop disk already which also offers a Live functionality.

Booted Ubuntu CD, and restored GRUB as sketched in this forum thread:

sudo grub
find /boot/grub/stage1
root (hd0,0)
setup (hd0)

Now at least GRUB was booting, but it also served me the next error message:

Kernel panic: VFS: Unable to mount root fs on unknown-block(0,0)

If tried to understand GRUB and the disk UUIDs mentioned in menu.lst, when I guessed that the problem was caused by a broken initrd.img.

I backed up the original initrd.img-2.6.24-16-server and copied the initrd.img-2.6.24-16-server.bak to the original name. And it worked!

As far as I can tell, the machine is back online and fully functioning again. But it was quite a trip ๐Ÿ˜‰


Extending default validity for self-generated SSL certificates

July 16, 2008

When I installed GForge recently, I had to generate an SSL certificate as part of the installation since GForge runs on https. However, the make-ssl-cert tool does not provide a way to customize the validity of the generated certificate, which is 30 days by default.

As it turns out, this problem is known for 3 years, and it is still not fixed in Ubuntu 8.04.

The easiest workaround is to edit the make-ssl-cert script using

whereis make-ssl-cert
sudo nano [path-to/]make-ssl-cert

and replace the line

openssl req -config $TMPFILE -new -x509 -nodes -out $output
    -keyout $output

with

openssl req -config $TMPFILE -new -x509 -days 365 -nodes 
    -out $output -keyout $output

That’s it.


Installing GForge on Ubuntu 8.04 Hardy

July 13, 2008

I was looking for a program that lets me easily provide download and versioning functionality for the software that I create, and I decided to give GForge a try on my Ubuntu server. Installation turned out to be a bit messy, but finally I managed to make it work.

First, I tried the gforge package that came with Ubuntu using apt-get, but it failed to install the database component gforge-db-postgresql with the message

constraint "forum_group_forum_id_fk" for relation "forum" already exists

(see bug report). From this post I figured out that several constraint triggers with the same name were created in the SQL script, which had been removed according to the changelog.

After modifying the SQL script, installation raised another error with a malformed SQL statement containing

WHERE oldvalue=100 AND field_name='category_id'

(sorry I lost the original error message). Unable (and also unwilling) to debug unknown SQL code, I decided to give up.

Next, I tried the gforge package provided by Christian Bayle. I added the repository to my apt sources, and started installation. Unfortunately, the same SQL error in the WHERE clause caused the installation to terminate unsuccessfully.

I uninstalled everything using apt remove, and turned to my last choice:

Manual installation

From the download page, I chose gforge-4.5.19, because it seems to be the most recent stable version. The installation manual can be found on the Documentation page (User Documentation / Gforge Manual (PDF)). Even though the manual was written for 4.5.3, it is still accurate enough. Chapter 2.3 covers installation.

Since the earlier installation attempts already created a PostgreSQL database, I had to drop both the database and the user

sudo su - postgres
DROP DATABASE gforge;
DROP USER gforge;

The tcpip_socket option in postgresql.conf is not recognized anymore (as of version 8.3).

Setting up the Apache configuration was a bit of a mess, because the previous attempts had generated a gforge.conf (/etc/gforge/httpd.conf) file which contained instructions that Apache2 does not recognize. They also added references to modules that do not exist, such as auth_gforge_module.

Anyway, executing a manual installation from scratch should definitely work with fewer problems than I experienced.

Finally, I got the web application to run, but I had not enabled redirection which caused to redirect an HTTP request to HTTPS. Therefore, when I clicked on the link “Register as a site user”, which requires an SSL connection, the browser display the error message:

(site URL) has sent an incorrect or unexpected message
(error code -12263)

There were strong indications that the error was related to SSL. ๐Ÿ˜‰

Fine, I thought, just activate the https redirection, enable the VirtualHost section in the apache config file, create an SSL certificate, and then we’re done. So, I locate the instructions to set up an SSL server, which rely on a script called apache2-ssl-certificate which is not installed on Ubuntu Hardy. Well, actually it is missing from Ubuntu versions for one and a half years now.

Fortunately, the work-around is in the comments:

sudo make-ssl-cert /usr/share/ssl-cert/ssleay.cnf
    /etc/ssl/private/gforge.pem

In the VirtualHost section, add (or uncomment) the lines

SSLEngine On
SSLCertificateFile /etc/ssl/private/gforge.pem

In the end, I could successfully register a new user, added the Admin privilege via PostgreSQL, and now I am the proud and happy owner of a gforge installation ๐Ÿ˜‰

Let’s see what adventures await me there.