Reinteract-ROX: a re-packaging tutorial

Earlier this month, Owen Taylor announced Reinteract ("a system for interactive experimentation with python"). I've been wanting something like this for a while now.

In this article, we're going to turn it into a ROX application. In the past, this has meant taking a copy of a program and renaming and changing things to fit the ROX application structure. The trouble is, the ROX version and the original upstream version diverge over time. So, we're going to look at how improvements in tooling can make things easier for us.

The first bit of good news is that Reinteract uses a GIT, an excellent decentralised version control system (which we also use for ROX-Filer). GIT allows us to clone Owen's repository, make any changes we want, and resynchronise at any time with a single command. This should be a big improvement over the situation a few years ago when people were using systems like CVS and Subversion. It's hard work synchronising two subversion repositories.

So, we'll start by cloning the original repository:

$ git clone git://git.fishsoup.net/reinteract

Metadata

We could start by moving things around to create a ROX application directory, but that will cause conflicts when we try to merge in new versions. It's also very unlikely that a desktop-neutral project is going to apply such changes, forcing us to maintain them indefinitely. Instead, we're just going to add some metadata saying where things are. The 0publish command can create a template for you and validate what you enter, or you can just type it in manually. Here's what I added, in a file called Reinteract.xml:

<?xml version="1.0" ?>
<interface xmlns="http://zero-install.sourceforge.net/2004/injector/interface">
  <name>Reinteract-ROX</name>
  <summary>a system for interactive experimentation with python</summary>
  <description>
Reinteract is a system for interactive experimentation with python.
You enter Python code and expressions and immediately see the results.
What distinguishes Reinteract from a shell (such as IPython or the builtin
interactive mode) is that you can go back and edit expressions you entered
earlier and the results will flow through the part of the worksheet after
the changed portion.</description>

  <homepage>http://www.reinteract.org/</homepage>

  <icon href="http://www.reinteract.org/images/reinteract-logo.png" type="image/png"/>

  <feed-for interface="http://rox.sourceforge.net/2007/interfaces/Reinteract.xml"/>

  <group license="OSI Approved :: GNU General Public License (GPL)" main="bin/uninst.py">
    <implementation id="." version="0.1-pre"/>
  </group>
</interface>

In writing this file, we're just restating the information on the Reinteract web-site: that this is a pre-release version (0.1-pre), that you run it using the command bin/uninst.py, etc. But, crucially, we're doing it using a well-defined structure that we can process automatically using our existing tools. The only ROX-specific changes were to rename the program to "Reinteract-ROX", so that people don't get confused about which version they're using, and to specify where I will publish it (in the feed-for element; essentially this is the web-page I will put up to describe my modified version).

Commit the changes to your local GIT repository (no network access is required for this):

$ git add Reinteract.xml
$ git commit -a

An easy way to test that the file is roughly right is to try running it with 0launch (which is probably in a package called "zeroinstall-injector" in your distribution). This command should run it as if you'd run the bin/uninst.py script directly:

$ 0launch ./Reinteract.xml

Finally, you should publish your GIT repository somewhere. I registered one at http://repo.or.cz/w/reinteract/rox.git, and added that to my local repository under the name "main":

$ git remote add main git+ssh://repo.or.cz/srv/git/reinteract/rox.git

Adding ROX-style saving

Next, we make our changes. I added drag-and-drop saving, which was rather straight-forward (see the changes on the rox branch of my repository if you're interested):

Note that my repository has two branches: "master" contains modifications that upstream may want to pull back, while "rox" contains the ROX-specific modifications.

Creating a release

Reinteract hasn't had an official release yet, so we'll make one now. Making releases is tedious and involves many steps if you want to do it properly (such as tagging the version in GIT and making sure that the archive you release can be generated exactly from that version). I like to create a script to make releases; that way I always follow the same process for each version. Luckily, we have a (new) tool for this too. Create a new directory (not under version control) for the release files and set it up, like this:

$ mkdir releases
$ cd releases
$ 0launch http://0install.net/2007/interfaces/0release.xml ../Reinteract.xml

Here, we use the 0release tool to create a release script. The argument is the path to the XML metadata file we created above. 0release uses this information automate some parts of the release. Have a look in the generated make-release script, which contains settings that are specific to you (such as the command to upload the release archive to your download server).

You can leave all of the settings blank except for the first one, which says where you're going to host the files. For myself, I like to automate as much as possible, so I used the following settings for sourceforge ("update-rox-site" is a custom command I use to push changes to the live site):

ARCHIVE_DIR_PUBLIC_URL=http://osdn.dl.sourceforge.net/sourceforge/rox
MASTER_FEED_FILE="$HOME/rox/htdocs/2007/interfaces/Reinteract.xml"
ARCHIVE_UPLOAD_COMMAND='ftp upload.sf.net'
MASTER_FEED_UPLOAD_COMMAND='svn ci -m "New release" "$@" && update-rox-site'
PUBLIC_SCM_REPOSITORY=main

Finally, run the script to create the release:

$ ./make-release
Releasing Reinteract-ROX
Snapshot version is 0.1-pre
Version number for new release [0.1]:

Notice how 0release uses the metadata file we created above to suggest the next version number for us (accept the default offered).

Next, 0release prepares a tarball for us. It will also generate a changelog from the GIT history automatically, as a starting point for a release announcement:

Wrote changelog from start to here as changelog-0.1

Candidate release archive: reinteract-rox-0.1.tar.bz2
(extracted to /home/talex/src/reinteract/releases/reinteract-rox-0.1 for inspection)

Please check candidate and select an action:
P) Publish candidate (accept)
F) Fail candidate (untag)
(you can also hit CTRL-C and resume this script when done)
Publish/Fail:

You can test that the archive works if you like, and then enter P<return> to publish it. This:

• Uploads the archive to sourceforge.net (which, annoyingly, requires some further interaction using your web-browser).
• Tags the version it released in GIT (as v0.1) and pushes the changes to your public GIT repository.
• Uploads a signed XML file (which is similar to the metadata file above, except that it gives the final download location and digest of the release archive, rather than a local directory). In my case, this is done by committing the XML to subversion and then running a custom script to check it out on the server, but you may use some other system, of course. This XML file is called a feed; like an RSS feed, people can subscribe to it to be notified of new releases.

Using the release

So, what have we achieved? So far, we've just performed a rather normal software release. In particular, the archive doesn't contain a ROX application directory.

But we can turn it into one automatically! Like so:

$ 0launch http://rox.sourceforge.net/2006/interfaces/Zero2Bundle.xml \
      http://rox.sourceforge.net/2007/interfaces/Reinteract.xml

The Zero2Bundle program takes an XML file produced by 0release and creates a ROX application directory from it. After confirming that you trust the GPG key that signed the XML feed, a ROX savebox appears letting you save your new ROX application. This has the usual ROX application layout, with an AppRun file, a .DirIcon, etc. You can run it, add it to your panel, drag files to the icon to load them just like a normal ROX program.

Because there's nothing ROX-specific about all this, it's quite possible that upstream authors will provide suitable XML files themselves: instant ROX applications for free! And, indeed, some of the programs in the ROX-All package were produced like this, by non-ROX developers. All we did was add the feed's URL to the script that generates the ROX-All release!

Of course, standard ROX application directories are a bit limiting. We don't get notified of new versions for one thing, and if there are multiple users of the computer they won't automatically share the code. No problem! The XML feed file can be used by other programs too. In fact, ROX users are more likely to drag the feed to AddApp to create a launcher.

The again, the feed file is desktop neutral. There are programs for GNOME, KDE and Xfce which will add the program to those desktops. Maybe people will use the feed to create Debs or RPMs too, although I don't know of any tools to do that yet.

Getting updates

The advantage of this setup is the ease of releasing updates. GIT can track both my local changes and upstream ones. The history displayed below shows that Owen has made a change since I branched, and also that I thought of something to add to both the master and ROX branches (I added it to master and then merged it to ROX):

To pull any new features in from upstream:

$ git-pull origin

Then run make-release to upload it as version 0.2.