Allen Brooker (AllenJB)

Planet PHP recently popped up a post by Ilia with a warning about the configuration of PHP using AddHandler instead of AddType.

Now I’m all for people publshing warnings about security issues, but they really should at least read the official documentation to check their information first.

In this case, to summarise the documentation:

• AddHandler is used for server-side content handling – it associates a handler with the specified content
• AddType is used for determining the Content-Type in relation to the client request (ie. the default Content-Type identified to the browser)
• BOTH obey multiple extensions, but a response can only have one Content-Type while a file can be handled by multiple Handlers.

This means that while, when using AddType, “test.php.gif” no longer works, “test.php.something” will (assuming .something doesn’t have an associated Type), because .php is the last extension encountered which has an associated Type. So the so called “fix” doesn’t really fix the problem at all.

Additionally, when using AddType, instead of the default Content-Type being text/html, it becomes “application/x-httpd-php”, which is technically incorrect and may result in your website not being viewable in browsers or by search engine bots.

If you really want to make it so that only files ending with .php are handled by PHP, then you should use SetHandler instead of AddHandler. In fact, this is what the current official PHP documentation recommends.

Today I discovered app-arch/cfv in the Gentoo tree. In addition to all the usual md5, sha1, etc formats, install the net-p2p/bittornado package (gtk use flag not required) and this utility can also handle hash checking against .torrent files, without the overhead of launching a full blown torrent client and then having to make sure said client can find all the files.

This is useful to me as I torrent on a remote server and download the files over SFTP/FTPS. Very occasionally something can go awry and I have the desire to hash check the files to see if it was an error with the original or something in my copy that perhaps occurred during transit.

With cfv, I can instead simply store the .torrent file alongside the files and then, when I want to check the files, run “cfv -f foo.torrent“

Finally got some time to play around with Amarok 2 (current git) now I’ve got it working. So Amarok did finally reimplement labels, but there’s currently nothing you can actually do with them. What I really want is the old smart playlists back – so I can create a playlist that includes all tracks from my collection which have label “foo”, but not label “bar” and have a rating >3, for example. This is already in Amaroks bug tracker as Bug #190667.

https://bugs.kde.org/show_bug.cgi?id=168781

--- Comment #13 from Daniel Dewald <TheCrasher gmx de>  2009-11-18 02:56:25 ---
Labels support was added and is available in the GIT Version. First stable
release with label support will be 2.2.2.

Woot!

Portage 2.2 has recently introduced a new feature that allows users to select what licenses they want to allow on their install. The interface still needs some work (for example, Portage needs to explain when it has “ignored” packages due to license during updates) so that users don’t, for example, get confused as to why their system suddenly wants icedtea instead of sun-jdk. Other than these fairly minor issues, it seems to be working well so far.

There’s an article on Gentoo Wiki that explains this new feature from a users point of view and how to tailor it to your individual tastes.

I recently had a disk start spewing media errors all over my kernel logs, as they occaisionally tend to do. I unmounted it and then started some basic investigation to see how bad the problem was. The first thing I checked was whether the partition table was still in tact, which oddly it appeared not to be. The disk was reporting it was a fraction of its previous total size, even in the BIOS.

I did some further investigation and this was when I discovered the “Host Protected Area” (HPA) feature had been activated. At a guess, either the kernel or the disk itself activated this feature to prevent further damage to the troubled areas, but if you don’t know about it and don’t do deep enough research to find out about it, it could be easy to assume that the data is unrecoverable.

As the wikipedia article linked above mentions, a bit of hdparm magic (in my case thanks to sysrescuecd) and your entire disk will be visible again, leaving you free to attempt some data recovery (I used ddrescue and the errsize was only a few hundred K – haven’t found which files yet, so it’s likely data I don’t care about or never use!)

Just a heads up to say that I’ve finally discovered why logwatch doesn’t work with metalog-1 any more – some bright spark changed the default log format. On the up side, it’s configurable, so the configuration to change it back has been added to the guide I wrote on Gentoo Wiki

OK. So this week I’ve been banging my head against the previously mentioned 5 year old bug in run-crons.

The problem is, for /etc/cron.{hourly,daily,weekly,monthly} we have the following requirements:
1) Scripts should ideally be run at a specified time.
2) Ideally the time at which the scripts run should be slightly different for each of the groupings to avoid unnecessarily loading the system and reduce potential conflicts in related scripts in different groups.
3) Scripts should never run twice within the same time period
4) If the scripts are not run for any reason in their given time period (eg. the system is offline), we want to run them at the next convenient opportunity
5) Even if the scripts are not run at their normal time for any reason, we want to run them at the normal time in the next time period
6) Because this is something very useful, we want to do this with as few resources or dependencies as possible on as many platforms as possible

So the solution the world came up with is run-crons. Which works. Most of the time.

The real headache comes because of daylight savings. Even if we don’t schedule to run in daylight savings, it leaves its mark in the form of an extra or less hour in the day that’s unexpected. This causes a problem because run-crons compares the timestamp on its “last run” spool file with the current time. Once this gets more than 24 hours old, we want to run the daily scripts again… except on daylight savings changes, when this figure is 23 or 25 hours. And we’re trying to do this in a shell script, using the most generic tools possible.

So we could use “3am” as the time to compare to – but then run-crons has to know this time. It also still brings up the issue of if the system was down for 20 hours, and comes back up at 11pm, do you want to run the cron.daily scripts for that day still? Then again at 3am the following day?

Brain damage ensues.

Frankly I’m beginning to think the world would be a better place if we did away with cron.{hourly,daily,weekly,monthly} altogether and just used cron.d. But even that still doesn’t resolve all the issues.

So I noticed that when we switched off the abomination that is daylight savings this weekend just gone, my systems which run on “GB” time rather than “UTC” executed their cron scripts twice. This despite the fact that, at a glance, they should execute at (or shortly after) 03:01, which should be safe.

Turns out this isn’t quite the case. There’s a 5 year old bug in Gentoo’s Bugzilla about this known issue and it appears to be down to the way run-crons manages its lock / last run files. I’m already running a run-crons patched for recursive directory scanning on some of my systems, so it wasn’t that big a step to basically do away with Gentoo’s supplied script altogether.

Now I’m running “run-crons-in” – a much simplified script that simply relies on the cron daemon itself for timing (which is how it should be, in my opinion). Here’s my script:

#!/bin/bash
# 2009-10-27 Complete rewrite of run-crons
# This version is designed to be called using @hourly @daily @weekly or @monthly
# With, for example: run-crons-in daily
#
# This version assumes:
# - We don't care if a previous script is still running
# - The cron daemon handles all timing in a sensible manner
# - Therefore no locking is necessary

AJB_DEBUG=0
AJB_TIMES=1
AJB_PTIMES=1
AJB_TIMEFORMAT='%Y-%m-%d %T'
BASE=$1

AJB_STARTTIME="`date +\"${AJB_TIMEFORMAT}\"`"
[ $AJB_TIMES -eq 1 ] && echo "Start time: ${AJB_STARTTIME}"

run_recursive() {
        CRONDIR="$1"
        [ $AJB_DEBUG -eq 1 ] && echo "Executing cron scripts in directory: $CRONDIR"
        for SCRIPT in $CRONDIR/* ; do
                if [[ -x $SCRIPT && ! -d $SCRIPT ]]; then
                        [ $AJB_DEBUG -eq 1 ] && echo "Executing cron script: $SCRIPT"
                        AJB_PSTARTTIME="`date +\"${AJB_TIMEFORMAT}\"`"
                        [ $AJB_PTIMES -eq 1 ] && echo "Script start time: ${AJB_PSTARTTIME}"
                        $SCRIPT
                        AJB_PSTOPTIME="`date +\"${AJB_TIMEFORMAT}\"`"
                        [ $AJB_PTIMES -eq 1 ] && echo "Script stop time: ${AJB_PSTOPTIME}"
                fi

                if [[ -d $SCRIPT ]]; then
                        run_recursive $SCRIPT
                fi
        done
}

CRONDIR="/etc/cron.${BASE}"

test -d "$CRONDIR" || echo "Directory does not exist or is not a directory: ${CRONDIR}"
set +e
test -d "$CRONDIR" && run_recursive "$CRONDIR"

AJB_STOPTIME="`date +\"${AJB_TIMEFORMAT}\"`"
[ $AJB_TIMES -eq 1 ] && echo "End time: ${AJB_STOPTIME}"

As you can see, it’s fairly basic – altho I’ve added some extra output for timing and debugging, simply because I felt like it. I’m currently testing this long term to ensure it works as expected, but hopefully I’ll adopt this for all my systems within a couple of months.

The full /etc/crontab entries are now:

@hourly   root    /usr/local/sbin/run-crons-in hourly
@daily    root    /usr/local/sbin/run-crons-in daily
@weekly   root    /usr/local/sbin/run-crons-in weekly
@monthly  root    /usr/local/sbin/run-crons-in monthly

(Yes, I realize that, at least by the man page, those are all going to run at midnight, but daily is the only set of scripts that ever does anything really stressful on the system I’m currently testing on, and it’s a quad core, and I really don’t care otherwise I wouldn’t be testing cron daemons on it)

I’m also taking the opportunity to use cronie in place of vixie-cron. There’s little documentation for this project, but as far as I can see it’s a drop-in replacement for / fork of vixie-cron from redhat that’s actively maintained.

I did take a quick look at the other cron daemons in the portage tree, but cronie appears to be the only one that’s actively maintained.

I finally got around to configuring logwatch to work with metalog, and wrote up the process as a guide. You can find it over on Gentoo Wiki.