Extension:Subversion

From 3DN

How do you like quoting pieces of code into a Wiki and later on chancing that code but forgetting to update your Wiki page that contains that code? That's right, that kind of sucks doesn't it. So in the process of designing/coding the next revolutionary thing, I really need some good tools to do so. It shouldn't be all that hard to do I imagine.

So let's get cracking. First we search the internet offcourse to see if somebody has done this before us. That would be annoying at best. A search reveals that the only thing that comes close to it is: [CodeReview] but that's not quite what we mean. So let's start reading here: [Writing mediaWiki extensions].

One of the things we see here is a nice way to put code inside Wiki pages:

require_once("$IP/extensions/XXX/XXX.php");
$wgXXXConfigThis=1;
$wgXXXConfigThat=false;

Well that's very nice indeed but on simply cut and pasting the fragment at first it didn't look quite as good! So we found an extension on mediawiki.org that's not in the standard distribution. Let's look it up and install it. Looking at mediawiki's [Special Version page] we immediately find out that the extension responsible for this is the [Syntax Highlight extension]. After having installed this, the above should look much nicer! Fortunately the SyntaxHighlight_GeSHi extension is part of the extension bundle we installed earlier so all we have to to is

• mwenext SyntaxHighlight_GeSHi.php

Not a bad result, but onwards! So the idea is to make an extension that will:

• implement an svnsource file="..." version="..." structure
• Check if the above named file/version is inside the mediawiki cache, if it is not, check it out of subversion and into the cache
• Write out a source lang="..." tag
• Write contents of the file now in the mediawiki cache
• Write out a closing source tag

It will be interesting to see whether the source tag output by our own extension will in turn be parsed by mediawiki to do the syntax highlighting.

To get the setup going, I will not be using the 'Debian Way' for this mediawiki extension as I would like to make the sourcecode for it available through my own subversion server and allow people to tweak it. The 'Debian Way' has some, Hmm shall we say, 'limitations' for as far as I can tell. These are:

• The mediawiki extensions all sit in /usr/share/mediawiki-extensions (class files and stuff)
• Part of the mediawiki extensions sit in /etc/mediawiki-extensions/extensions-available
• Enabling the mediawiki extensions using mwenext enables the extension on all Wiki servers it seems. I'm not particularly fond of that

So I'll start building this extension in the standard (AFAIK) mediawiki way and simply create a subdirectory under /var/lib/mediawiki/extensions called 'WikiSubversion' and check this directory into my subversion server so I can do version control as I go.

So we start out with creating the extension very simply by creating Subversion.php in the /var/lib/mediawiki/extensions/WikiSubversion subdirectory with contents:

$wgExtensionFunctions[] = "wfSubversion";
 
function wfSubversion() {
    global $wgParser;
    # registers the <subversion> extension with the WikiText parser
    $wgParser->setHook( "subversion", "renderSubversion" );
}
 
function renderSubversion( $input, $argv ) {
    return $output;
}

I know, it could use some comments but let's keep real here, The above isn't even viewable by the subversion extension itself yet so why bother? I'm sure the contents above will change so it's just for posterity at this point.

Now we simply add a line:

require_once "$IP/extensions/WikiSubversion/Subversion.php";

to /etc/mediawiki/LocalSettings.3dn.php and verify whether mediawiki sees the extension in the Version page. How disappointing, it does not! Perhaps doing this the mediawiki way does require us to restart the Apache server.. This is interesting, after restarting Apache something does show up in the Special page saying something about:

<subversion>

Maybe the <subversion> tag was there before already... However, we don't see a nice label saying what's the name of the installed plugin. Some more digging around shows us that all we need to do to fix the above is to simply add:

$wgExtensionCredits['other'][] = array (
    'name' => "Subversion",
    'version' => "1.0, May 3rd, 2009",
    'author' => 'Fred Leeflang',
    'url' => 'http://fredl@3dn.nl',
    'description' => 'Subversion extension'
    );

Alright, since the extension shows up nicely now, let's make it print out some <source> tag and some code and see if that works.

<subversion></subversion> (attempt 1)

So that first attempt didn't work so well... I let the wfSubversion() function print out a source open and close tag with a simple 'print "Hello World!\n"' in between. Didn't display a thing! So then I make the wfSubversion() function simply print out a 'Hello World!\n' and it also didn't work. Restarting Apache didn't make a change... Oh wait a minute, could it be this pesky cache thingie again? Took out the <subversion></subversion> part, reloaded, reeditted and boom! the first 'Hello World!' from my script was born. Let's try this trick again on printing out some real code and then start digging into the caching function some more because that's starting to get pretty annoying.

<subversion></subversion> (attempt 2)

Aha, that works, the extension indeed outputs the desired <source> tags and some code inside. But... it also answers our earlier question that a tag outputted by an extension will not be parsed by another extension! This could mean trouble!

So let's dig into some other sources of information on this one. We visit IRC on the freenode network and go to #mediawiki:

<fredl>    I'd like the outputted source code to be inside <source></source> tags but those don't seem 
           to get parsed when output by an extension. Is there any way to do that?
<Splarka>  hmm, that should depend on what hook you use, are you making your own <tag> or your own {{#parserfunction}}, or 
           some other method?
<fredl>    I'm making my own <tag>.
<Splarka>  okay, the output of tag bypasses the main parser, the main use for this is to create things the parser might 
           not natively allow
<Splarka>  such as unwhitelisted html, or preserved whitespace
<fredl>    (I'm not sure I've seen the Special:Code extension)
<Splarka>  your extension might work if you simply use {{#tag:tagname|data}}
<Splarka>  but you might consider rewriting it as a parser function
<Splarka>  http://www.mediawiki.org/wiki/Manual:Parser_functions

<fredl>    Splarka, another question I had was... the extension should first check in the cache whether it has already 
           checked out a local copy of the indicated SVN file.
<fredl>    if it doesn't have a copy, it should check out the file to the cache, embed it's contents between the 
           <source></source> tags
<fredl>    does that sound do-able?
<fredl>    I saw some cache functions in the doc...
<fredl>    but none that I thought looked like good candidates to search through a hash table or something 
           (very new to extension coding)

<Splarka>  mmm, all greek to me, but check out http://www.mediawiki.org/wiki/Extension:CodeReview
<Splarka>  does something similar (fetches diffs and caches them)
<fredl>    ah okay... I had looked into that extension before but didn't think it did what I want to do, but I can at 
           least take a look at the code, that'd be usefull.
<fredl>    thanks!

So very useful, IRC. The Special:Code page looks very nice indeed. Even though it doesn't seem to do exactly what I want it to do I'm going to install it anyway, just to get a feel for what the extension is doing.

A resource even more useful than IRC is ofcourse to check what's there already and while searching for the Special:Code page I ran into [this gem]. It does exactly what I want to do! So I'm going to install that extension and let's see.

To install SVNIntegration I first had to do some other things on my system:

• First I had to install the Debian package php-pear. Easy enough, apt-get install php-pear
• After pear's installed I needed to use it to install the package VersionControl_SVN-0.3.1: sudo pear install VersionControl_SVN-0.3.1

Very nice indeed and very painless. So... onwards.

<google uid="C08" position="right"></google>