Cats and Code » perl http://blog.gorwits.me.uk by Oliver Gorwits Sat, 29 Mar 2014 23:28:44 +0000 en-US hourly 1 http://wordpress.org/?v=3.6.1 Hacking # in OS X http://blog.gorwits.me.uk/2011/11/30/hacking-in-os-x/?utm_source=rss&utm_medium=rss&utm_campaign=hacking-in-os-x http://blog.gorwits.me.uk/2011/11/30/hacking-in-os-x/#comments Wed, 30 Nov 2011 20:57:05 +0000 Oliver Gorwits http://blog.gorwits.me.uk/?p=717 Continue reading ]]> To get a # sign on an Apple keyboard you use the Option (or Alt) key + 3. This seems terribly klunky to me, and # is of course used quite a bit in programming and sysadmin work.

This hack remaps another key on the keyboard to produce the # character. I chose the funny squiggle that’s to the left of the number 1 key (§). This is the Section sign, used in document formatting. Just create a file at ~/Library/KeyBindings/DefaultKeyBinding.dict which contains the following:

{
    /* this will make all § turn into # */
    "\UA7" = ("insertText:", "#");
}

Any app that uses Apple’s Cocoa interface widgets for text input will pick this up after being restarted. There are some that don’t (perhaps TextMate? Not checked that one so if you know, please comment).

A lot more information about this is available at this excellent page on the Cocoa Text System, including some other neat hacks. Enjoy!

]]>
http://blog.gorwits.me.uk/2011/11/30/hacking-in-os-x/feed/ 0
A (very) short list of Dist::Zilla tips http://blogs.perl.org/users/oliver_gorwits/2011/11/a-very-short-list-of-distzilla-tips.html?utm_source=rss&utm_medium=rss&utm_campaign=a-very-short-list-of-distzilla-tips http://blogs.perl.org/users/oliver_gorwits/2011/11/a-very-short-list-of-distzilla-tips.html#comments Thu, 24 Nov 2011 21:43:41 +0000 Oliver Gorwits Continue reading ]]> For App::fooapp type distributions then you might want the README etc generated from a specific file. Add this to dist.ini:

main_module = bin/fooapp
; btw, semicolon leads a comment, in case you forgot how to do that

Any Module::Install converts reading this should note the bin directory, not script.

In bin/fooapp itself you also provide an additional metadata hint (next to ABSTRACT):

# PODNAME: fooapp

Which results in nifty Metacpan links such as:

Finally this hint to the POD munger will allow a section to be pinned in place above the SYNOPSIS:

=begin :prelude

# POD here...

=end :prelude

I hope these tips are helpful to some…

]]>
http://blog.gorwits.me.uk/2011/11/24/a-very-short-list-of-distzilla-tips/feed/ 0
Painless MythTV Channel Configuration http://blog.gorwits.me.uk/2011/11/10/painless-mythtv-channel-configuration/?utm_source=rss&utm_medium=rss&utm_campaign=painless-mythtv-channel-configuration http://blog.gorwits.me.uk/2011/11/10/painless-mythtv-channel-configuration/#comments Thu, 10 Nov 2011 23:07:46 +0000 Oliver Gorwits http://blog.gorwits.me.uk/?p=687 Continue reading ]]> MythTV – a brilliant homebrew digital video recorder system. Killer features include being able to play content over the LAN at home, scheduling recordings via the web, and generally poke it to integrate with all kinds of devices (e.g. see my previous posts on H.264 transcoding). Even better, Mythbuntu makes installation a doddle.

However the most hated part for me is configuring TV sources and channels – digital terrestrial via an aerial, and digital via satellite. MythTV’s built-in scanner works at best intermittently (for me), and when it does, comes up with 1,000 shopping and adult channels which drown out the 20 or so I’m really interested in.

Then there’s TV listings. All credit to the folks working on XMLTV and the Radio Times listings grabber – that’s some impressive work. But stitching it into MythTV usually ends up with hand-editing the database to insert XMLTV IDs. User friendly? I think not.

Partly this is because these tools are used internationally and nothing is standardised between countries. Even in the UK there are three ways to get TV listings (EIT over the air, Bleb, and Radio Times).

Finally I snapped, and wrote a Perl program to do all this work. It feels so nice now to have a simple, lightweight, repeatable process to configure sources and channels. That’s what good automation is all about.

The code will only work in the UK, but might be a starting point for those elsewhere. It configures XMLTV IDs, but that doesn’t mean you have to use the Radio Times grabber. You still have to go through MythTV’s setup program to tell it about tuner cards (before running the import program) but that’s not hard work.

The code and instructions are hosted on GitHub. Let me know if you use it, and how you get on. Don’t forget to back up your database (using MythTV’s mythconverg_* scripts) before starting!

]]>
http://blog.gorwits.me.uk/2011/11/10/painless-mythtv-channel-configuration/feed/ 0
Hosting the AutoCRUD Demo http://blog.gorwits.me.uk/2011/10/19/hosting-the-autocrud-demo/?utm_source=rss&utm_medium=rss&utm_campaign=hosting-the-autocrud-demo http://blog.gorwits.me.uk/2011/10/19/hosting-the-autocrud-demo/#comments Wed, 19 Oct 2011 18:19:59 +0000 Oliver Gorwits http://blog.gorwits.me.uk/?p=680 Continue reading ]]> In my previous entry here (syndicated from blogs.perl.org), I linked at the end to a demo Catalyst::Plugin::AutoCRUD application running on DotCloud. I’m much happier with this than running something on my own personal server, and here’s the notes on its setup.

For those unfamiliar, DotCloud is a Platform as a Service (PaaS) offering a freemium model. I’m grateful to them for this as the free account provides all I need for my demo.

First, I followed to the letter Phillip Smith’s comprehensive guide on deploying a Perl Catalyst application to the DotCloud service. Next I customised the basic application created in the guide to use AutoCRUD:

  • removed the Root controller
  • added two Models and their supporting DBIx::Class Result classes
  • set basepath in the configuration
  • installed an hourly cron job to:
    • restore the SQLite databases
    • restart the web service (supervisorctl restart uwsgi)

Next I wanted a more tidy looking domain for the demo, so purchased autocrud.pl through NETIM. My plan is to have demo.autocrud.pl pointing to the DotCloud instance, and sometime in the future to have autocrud.pl be used for a secret feature I’m still working on. Sadly NETIM only offers HTTP redirects from subdomains, so I delegated hosting of the DNS to ClouDNS.

ClouDNS is another freemium service, again where the free part provides just what I need. They offer not only a bit of a smarter interface than NETIM for DNS zone management, but also HTTP redirects from the zone apex.

I do of course know that nothing lasts forever, particularly with freemium services, and I’m grateful for what’s available because it works very well (I’ve added promotional icons for ClouDNS and DotCloud to the demo site).

The end result of this is that I now have the AutoCRUD demo safely hosted on DotCloud with a friendly URL to pass out in documentation or blog posts :-)

]]>
http://blog.gorwits.me.uk/2011/10/19/hosting-the-autocrud-demo/feed/ 0
AutoCRUD revamped http://blogs.perl.org/users/oliver_gorwits/2011/10/autocrud-revamped.html?utm_source=rss&utm_medium=rss&utm_campaign=autocrud-revamped http://blogs.perl.org/users/oliver_gorwits/2011/10/autocrud-revamped.html#comments Mon, 17 Oct 2011 20:47:54 +0000 Oliver Gorwits Continue reading ]]> For a couple of years I’ve been planning to rip apart and put back together the guts of Catalyst::Plugin::AutoCRUD, to address limitations in the initial implementation. After changing job and moving house I’m pleased this finally came to the top of my hacking stack.

Nothing was going to happen however before I could work out how to do one thing: achieve independence from DBIx::Class as a “storage engine”. I love DBIx::Class, but it would be much more cool to support any data storage system able to represent a table+column paradigm (even things like CSV, as a test case).

So SQL::Translator hit me like a thunderbolt. Of course, that’s exactly what it does - introspect some data storage and provide a neutral, class-based representation of the tables and columns (fields). It’s a little rough around the edges, but certainly good enough. The Translator provides a metadata structure which AutoCRUD’s web front-end can use, independent of any particular storage engine such as DBIx::Class. This also paves the way for development of display engines other than the bundled ExtJS and simple HTML offerings.

Right now there’s a developer release of AutoCRUD on CPAN, and I hope shortly to have a production release. Whilst the web side might not look much different, the fact is that it can now support significant features such as tables with composite/compound primary keys, or no primary keys for that matter, database views, relations to self, multiple relations to the same table, and so on.

Alongside that, I’ve taken the opportunity to fix a few quirks of the web interface, and chomp my way through the outstanding wishlist. The updated code is now running on a DotCloud instance, so please go and have a play!

p.s. a cron job will restore the demo’s databases at the top of every hour

]]>
http://blog.gorwits.me.uk/2011/10/17/autocrud-revamped/feed/ 0
Releasing trial/dev/beta versions with Dist::Zilla http://blogs.perl.org/users/oliver_gorwits/2011/10/releasing-trialdevbeta-versions-with-distzilla.html?utm_source=rss&utm_medium=rss&utm_campaign=releasing-trialdevbeta-versions-with-distzilla http://blogs.perl.org/users/oliver_gorwits/2011/10/releasing-trialdevbeta-versions-with-distzilla.html#comments Fri, 14 Oct 2011 07:17:01 +0000 Oliver Gorwits Continue reading ]]> You might have stumbled across Dist::Zilla's --trial command line option in the past, and maybe even used it for a developer CPAN release. Its effect is (as I understand it) two-fold:

  • adds -TRIAL to the name of the distribution archive being produced
  • sets release: testing in the META.json file which is parsed by CPAN services

It came to my attention that using -TRIAL is actually pretty bad for you and your system, and other users, even though it's one of the two naming conventions CPAN services use to identify developer releases.

The problem is that the actual $VERSION of your code is unaffected. This means once installed, you can't ask your computer the version of an installed distribution and work out from that whether it's a developer release, or not. A secondary issue is that in sites such as metacpan.org there's nothing really obvious about the release which highlights its status as "development", in the list of available versions.

An alternative way to signal to CPAN services that a dist is a trial release is to use an underscore and a secondary version number at the end of $VERSION, like _001. This is still a bit crappy but at least humans can really easily see what's going on.

Back to Dist::Zilla. If you use the AutoVersion plugin, a better alternative than using --trial is to set the DEV environment variable when you build or release the distribution. This has the effect of:

  • (sprintf '_%03u', $ENV{DEV}) being added to the end of $VERSION
  • sets release: testing in the META.json file which is parsed by CPAN services

Otherwise the best thing to do right now is to set the version manually, for developer releases. I hear from chatter on IRC that there are plans to change the --trial feature of Dist::Zilla to alter $VERSION if necessary (that is, if no underscore exists) - a good compromise, I reckon.

]]>
http://blog.gorwits.me.uk/2011/10/14/releasing-trialdevbeta-versions-with-distzilla/feed/ 0
local::libs for Dist Development http://blogs.perl.org/users/oliver_gorwits/2011/07/locallibs-for-dist-development.html?utm_source=rss&utm_medium=rss&utm_campaign=locallibs-for-dist-development http://blogs.perl.org/users/oliver_gorwits/2011/07/locallibs-for-dist-development.html#comments Sat, 16 Jul 2011 15:26:25 +0000 Oliver Gorwits Continue reading ]]> Most of my distributions are on GitHub and built using Dist::Zilla. As the dependencies of each vary widely and I don’t want to muck up my workstation’s libraries, I set up a local::lib for each distribution’s development.

The App::local::lib::helper scripts make this really easy. As per the docs, I combine the helper with App::cpanminus (cpanm) for all installation.

To bootstrap a new local::lib area, I wrote this simple shell script:

#!/bin/bash
# script named "new-ll"

if [ -z $1 ]
  then
    echo 'pass the distribution name, please'
    exit
fi

echo "creating local::lib for $1 ..."
sleep 3

curl -L http://cpanmin.us/ | perl - --notest --quiet --local-lib \
    ~/perl5/$1 \
    App::cpanminus \
    Dist::Zilla \
    App::local::lib::helper

.
Entering the correct environment for a distribution uses another helper script:

#!/bin/bash
# script named "go"

if [ -z $1 ]
  then
    echo 'pass the distribution name, please'
    exit
fi

~/perl5/$1/bin/localenv bash

.
Which means my workflow for a new distribution is:

$ new-ll New-Dist-Name
$ go New-Dist-Name

.
Any Perl distributions installed in that shell (for example from dzil authordeps | cpanm or dzil listdeps | cpanm) will be placed into the new local::lib. It’s a simple ^D to exit.

However it’s not obvious that you’re within this special environment, so editing Bash’s $PS1 variable (the shell prompt) to include the following, can help:

echo $PERL5LIB | cut -d'/' -f5

.
My deep thanks to the authors of the distributions used to create this neat setup.

]]>
http://blog.gorwits.me.uk/2011/07/16/locallibs-for-dist-development/feed/ 0
Perl on a Windows 7 laptop http://blogs.perl.org/users/oliver_gorwits/2011/04/perl-on-a-windows-7-laptop.html?utm_source=rss&utm_medium=rss&utm_campaign=perl-on-a-windows-7-laptop http://blogs.perl.org/users/oliver_gorwits/2011/04/perl-on-a-windows-7-laptop.html#comments Mon, 18 Apr 2011 20:33:58 +0000 Oliver Gorwits Continue reading ]]> I’ve been setting up my Windows 7 laptop to have as complete a Perl development environment as possible. I don’t have a choice about the operating system in this case, but I am used to the Unix-style environment, which I’d like to maintain.

One option is to use Cygwin and just dive into there for everything. However I do want some of my code to work natively under Windows so there is still a need to run Perl tests at a Windows console. I might as well develop there too, if I can. I’ve ended up with the following steps, which are in places bodgy hacks, but show the principle works at least:

  1. Git for Windows
  2. Strawberry Perl
  3. Vim fix
  4. Windows Console
  5. perldoc fix
  6. local::lib
  7. cpanm installer
  8. Local CPAN mirror

First thing is to install Git for Windows, known as msysgit (don’t alter any of the Windows PATH settings at install time). It’s a great little system which ships with Bash, Vim, and a suite of other handy tools. On this platform you probably want to set the End Of Line character(s) to ensure consistency. I have the following in my ~/.gitconfig:

[core]
    editor = vim
    eol = lf
[color]
    ui = true

The bundled Perl in msysgit is a little ancient (5.8.8). Download and install the excellent Strawberry Perl distribution. Our goal is to have perl within the msysgit environment be Strawberry Perl’s perl. Move C:\Program Files\Git\bin\perl.exe out the way (to perl-g.exe) and copy C:\strawberry\perl\bin\perl.exe in its place (there are no symbolic links under Windows).

By changing the Perl though, you knacker parts of Git which use Perl (for example git commit --interactive). The way to fix this is to keep a copy of the original msysgit perl (as above) and then edit any of the scripts in C:\Program Files\Git\libexec\git-core to have #!/usr/bin/perl-g at the top. Here’s a list:

  • git-add--interactive
  • git-svn
  • git-difftool
  • git-relink
  • git-send-email

The msysgit bundled Vim also needs some help, in the form of syntax files. I downloaded and installed GVim for Windows, and copied the .vim files from C:\Program Files\Vim\vim73\syntax to C:\Program Files\Git\share\vim\vim73\syntax but perhaps there’s a less brute-force way. At the least, you should get hold of perl.vim and pod.vim.

The Windows console program leaves a lot to be desired. I found a reasonable alternative in the form of Console2, an open source project hosted on Sourceforge. Download and install that (well, copy to Program Files). You’ll definitely want to edit its Settings so under Hotkeys, set Close tab and Rename tab to an alternative, or None (by doing Clear then Assign for each). Also under Settings in the Console section I set the Shell to be C:\Program Files\Git\bin\sh.exe --login -i and the Startup dir to be a folder I created in my user area for Perl development. Now, pinning the Console2 application to the Start Menu means it can be opened straight into msysgit’s Bash.

At the shell I installed my own vimrc file to ~/.vimrc, and created a ~/.bash_profile containing the following:

alias ll="ls -l --color"
alias view="vim -R"
export TERM=vt100

Strawberry Perl ships with many useful tools in its bin directory. Some of them are Windows batch files (.bat) because the expectatation is that you’re running Perl from the native Windows console. In this setup we’re within a Bash shell so they don’t work. One thing you’ll probably miss is perldoc, so create a ~/bin directory and in ~/bin/perldoc put the following:

#!/usr/bin/perl
require 5;
BEGIN { $^W = 1 if $ENV{'PERLDOCDEBUG'} }
use Pod::Perldoc;
exit( Pod::Perldoc->run() );

I always use a local::lib area for the modules used in my development, and Strawberry Perl ships with this module. Simply run the llw32helper program from the normal Windows console and it’ll ask you for the new local::lib location then set Registry entries appropriately. One reboot later and perl -V should include this new library path.

From Strawberry Perl’s CPAN shell (perl -MCPAN -e shell) I only install one module - App::cpanminus. This provides a new tool, cpanm, for installing modules. Note that since the previous local::lib setup, App::cpanminus is installed to that location rather than Strawberry Perl’s standard library path.

The final step is to have a local CPAN mirror. This means I can work on the Windows 7 laptop from a remote part of a Scottish Hebridean island, for instance, fully off-line. This uses the CPAN::Mini and CPAN::Mini::Webserver modules, both installed via the cpanm utility. The minicpan command line app will create the mirror for you. The minicpan_webserver app fires up an interface at http://localhost:2963/. Here are example commands used with this setup, which I’ve made into Bash aliases (also set your ~/.minicpanrc as per the docs):

minicpan -l /path/to/local/minicpan -r http://your.cpan.mirror.example.com/
cpanm --mirror file:///C:/path/to/local/minicpan --mirror-only -v Module::Name

A remaining niggle is that cursor keys don’t seem to work in less (Ctrl-N and Ctrl-K are alternatives). I’ve not yet found a fix for this; however they do work in Vim. Another issue I found is that, bizarrely, cpanm sometimes fails unless the -v option is provided. Also with cpanm it seems more successful at unpacking dist tarballs when treating the local platform as UNIX rather than WIN32 so I hacked the script to do that. I wish I were more capable on this platform to begin to debug this (please email me if you want to collaborate).

That’s pretty much where I’m at, right now. I’d welcome constructive feedback on this process in the comments (at blogs.perl.org if you’re reading a syndication). My deep thanks and respect to all the developers of the above tools and libraries, who have made this possible.

]]>
http://blog.gorwits.me.uk/2011/04/18/perl-on-a-windows-7-laptop/feed/ 0
Dist::Zilla::PluginBundle::Author:: namespace http://blogs.perl.org/users/oliver_gorwits/2011/02/dist-zilla-pluginbundle-author-namespace.html?utm_source=rss&utm_medium=rss&utm_campaign=distzillapluginbundleauthor-namespace http://blogs.perl.org/users/oliver_gorwits/2011/02/dist-zilla-pluginbundle-author-namespace.html#comments Sat, 05 Feb 2011 19:20:16 +0000 Oliver Gorwits Continue reading ]]> Following the lead of Mike Doherty (DOHERTY) I've moved my own Dist::Zilla Author PluginBundle into a new namespace:

Dist::Zilla::PluginBundle::Author::OLIVER

Having Author PluginBundles is a great system for saving me time and allowing others to build my modules with ease. However I had to agree to Mike's point that polluting the Dist::Zilla::PluginBundle:: namespace wasn't so cool. It also makes things more clear in the dist.ini file:

[@Author::OLIVER]

If you have an Author PluginBundle, please consider moving it into this new namespace. You can of course keep the old one around, in parallel, until code using it has moved off to the backpan.

]]>
http://blog.gorwits.me.uk/2011/02/05/distzillapluginbundleauthor-namespace/feed/ 0
Adding a new RANCID device class http://blog.gorwits.me.uk/2011/02/04/adding-a-new-rancid-device-class/?utm_source=rss&utm_medium=rss&utm_campaign=adding-a-new-rancid-device-class http://blog.gorwits.me.uk/2011/02/04/adding-a-new-rancid-device-class/#comments Fri, 04 Feb 2011 12:21:18 +0000 Oliver Gorwits http://blog.gorwits.me.uk/?p=312 Continue reading ]]> The RANCID system is used by many network service providers to backup and audit their network device configurations. It supports many device vendors (Cisco, Juniper, etc) but you might run into a vendor which is not supported. Adding a new device class (in other words, a vendor) to RANCID is not difficult.

Starting with the basics, we need a script which will connect to the device and emit the output of any commands you want to run there. You could clone and modify one of the existing RANCID scripts such as /usr/bin/jrancid or write your own. Either way, when run it needs to:

  • Accept as its final command line argument the name of the device
  • Save output to a file in the current working directory, called <devicename>.new

Install this script into /usr/bin/... with a suitable name.

The next step is to make RANCID aware of this new class of device. Load up the file /usr/bin/rancid-fe in your editor, and scroll down to see the lookup table for all device classes. Add an entry in there which has a “friendly name” as the key on the left, and the name of your script (created in the previous step) as the value on the right.

That’s it, you can now use this device class (the “friendly name”) in your router.db files and RANCID will execute your script to retrieve the configuration. You aren’t limited only to devices with a CLI now, of course. Your script could retrieve configuration (or any other text data) from any source by any means and save it to the .new file.

]]>
http://blog.gorwits.me.uk/2011/02/04/adding-a-new-rancid-device-class/feed/ 0