Compiling ejabberd for Ubuntu Lucid server

But there’s an apt package

Yes. But it kind of sucks. I had problems with the bundled init.d script. It tries to start ejabberd as the user “ejabberd” but something goes awry. It seems to start but I’m then unable to connect to that node using the ejabberdctl command. Maybe its just me? meh.

What about the Process One binary installer?

That works if you’re content with running ejabberd under root user or whatever user you installed ejabberd as. If installed as a user other than root you’ll have a problem trying to start ejabberd with any user other than root. Thats because the .erlang-cookie file is expected in the $HOME directory of the user starting ejabberd. Its a file permissions nightmare. bleh.

Running ejabberd under a safe(r) user account

Install dependencies

Lets use apt to install the dependencies for ejabberd. Its just a shortcut to getting the dependencies. You can do it the hard way. If you want

apt-get build-dep ejabberd


Download ejabberd source

We’ll get the source for ejabberd 2.1.3 the lazy way too. You could of course clone the git repo and checkout version 2.1.3 but a clone can take a while over a slow-w-w-w connection.

wget http://github.com/processone/ejabberd/tarball/v2.1.3 && tar -xzf processone-ejabberd-v2.1.3-0-g962b66b.tar.gz && cd processone-ejabberd-84987d6/src


Creating a user to run ejabberd

This is important. We’ll need it for the ./configure command coming later


sudo groupadd --gid 107 ejabberd
sudo adduser --system --no-create-home --uid 107 --gid 107 --disabled-password --disabled-login ejabberd


Note: I’ve used the same uid and gid in the line above. uid 107 may already be in use on your system. You can check for uid‘s already in use by grep’ing your /etc/passwd. Ensure the numeric value for uid and gid is the same as shown above. I’m not so sure the next step will work as intended if its not.

Configure (with something special), make, make install

Here’s the part that really matters

cd /path-to-untarred-ejabberd/src
sudo ./configure --prefix /usr/local/ejabberd-2.1.3/ --enable-user=ejabberd --enable-odbc --enable-full-xml
sudo make
sudo make install


There are two other options available for the ./configure script. Check this out. You may want MSSQL Server support or whatever. The only must-have for the purpose of this install method is the –enable-user option. The rest can be changed to your liking

The –enable-user option allows ejabberd to be started by root and/or the user account specified. Permissions will be set accordingly by the install process

A sane init.d script

ejabberd is now installed but you’d like it to automatically be started on system startup? You’ll need an init.d script for that and I have one for you.

I mauled the init.d script shipped with Process One‘s binary installer to my liking. Download revised init.d script from github. Place it in your /etc/init.d/ folder naturally. It works. Trust me already

Make ejabberdctl executable from anywhere

This is a lazy hack you could do it “right” by modifying your $PATH env variable in /etc/profile for all users or whatever

sudo ln -s /usr/local/ejabberd-2.1.3/sbin/ejabberdctl /usr/sbin/ejabberdctl


Starting/stopping ejabberd

The ejabberd process will automagically be daemonized erlang-style and the ownership of the ejabberd process is handed over to the ejabberd user we created earlier

root@blahblah:~# /etc/init.d/ejabberd start
Starting ejabberd...
done.

ps aux | grep ejabberd
ejabberd 21200 0.0 19.5 207044 49004 ? Sl 18:07 0:01 /usr/lib/erlang/erts-5.7.4/bin/beam.smp -K true -P 250000 -- -root /usr/lib/erlang -progname erl -- -home /usr/local/ejabberd-2.1.3/var/lib/ejabberd -- -sname ejabberd@localhost -noshell -noinput -noshell -noinput -pa /usr/local/ejabberd-2.1.3/lib/ejabberd/ebin -mnesia dir "/usr/local/ejabberd-2.1.3/var/lib/ejabberd" -s ejabberd -sasl sasl_error_logger {file,"/usr/local/ejabberd-2.1.3/var/log/ejabberd/erlang.log"} -smp auto start
root 21257 0.0 0.2 6152 660 pts/5 S+ 18:37 0:00 grep --color=auto ejabberd


Confirmation that the ejabberd node has been started

root@blahblah:~# ejabberdctl status
The node ejabberd@localhost is started with status: started
ejabberd 2.1.3 is running in that node


And that is all. You could do the same for ejabberd 2.1.4 (I reckon). I didn’t install 2.1.4 because its a little too bleeding edge but you decide for yourself

The real reason why Flash is not on the iPhone,iPad and iPod Touch

Flash is slow (well… yes kind of… maybe)

Sure its true that flash can be a hog. Flash optimists will proclaim that developers that write bad code are the ones giving flash a bad name. Who knows?

Flash can’t do hover (some say)

Of late, the latest and greatest is reason is that flash doesn’t support the hover event or something. And just in a rebuttal claiming that reason is moot. Doesn’t matter if this is true or not. I think we’re all beating around the wrong bush.

Steve Jobs is deflecting

IMO all the technical reasons for why flash is not on the iPhone and iPad are simply convenient excuses which deflect from the REAL reason why flash is not available on the iPhone/iPad.

The Real Reason

Flash apps would compete with the app store. It’s that simple. If flash was allowed on the iphone how much developers would write flash apps instead of native iphone apps? Plenty

Apple make their money from the apps not the devices (although the early iPhone adopters were gutted for $400 a piece)

Eventually most manufacturers will be capable of making a smartphone that is comparable to that of the iPhone’s software/hardware. Q: Which smartphone will people buy?

A: The one with the biggest 3rd party app platform

Developers will flock to the platform with most the users and users will flock to the platform with the most apps. Its a network effect

The real reason is strategic not technical

And then there’s the iPad CPU

iPad uses a proprietary Apple-designed CPU which means you can’t run linux or any other OS on your iPad nor can you hackintosh your intel netbook with the iPad OS. Could it be that this is also a strategic business decision and not a technical one?

Agile Adoption: When the going gets tough

Success abounds

The intertubes is home to a multitude of blog posts, videos, prezzos,case studies evangelizing the agile method for software project management. Many cite phenomenal results with agile and so-called hyper-productive teams. It all sounds too good to be true because it kind of is. What’s the catch?

Agile is simple but hard

Agile processes are deceptively simple but the underlying principles require much understanding. Take a look at scrum for a moment, a very popular gateway to agile adoption. The official scrum guide emphasizes processes but not so much agile principles. Don’t get me wrong. Scrum has not abolished agile principles but instead appears to be tailored for for shu-level agile practitioners.

Starting out is easy

As you build your product it becomes increasingly more complex (hopefully not complicated) resulting in the need for increasingly complex decision-making. Initially its very easy to follow the Scrum recipe for agility but as we progress from iteration to iteration we need to be able to respond to the changing environment.

If you’re still blindly following the scrum recipe by the book (i.e. the process) 20 iterations on you’re in big trouble. You need an understanding of the underlying principles. You need to progress from shu-level. Fast.

Be agile about your agile adoption

If you’ve adopted agile for a particular project realize that you now have two projects. The software you’re developing and your team or organization’s state of agile adoption.

Retrospectives are crucial

Always have a retrospective meeting at the end of an iteration. In fact make it a rule that an iteration is incomplete without such a meeting. Not only is the retrospective a reflection on the latest iteration but its also function as an inspection point for the state of your team’s agile adoption. Ask yourself is agile working for you or are you working for it? Every retrospective should identify at least one problem. I doubt any team has ever had a perfect iteration. This is an opportunity for self-reflection, correction, inspection, adaptation blah blah.

How to tell if you’re failing

Things might be going well for your agile/xp/scrum team. Don’t be deceived. Just like a code smell is an indicator of a larger problem in a codebase so can seemingly small issues be a manifestation of greater ones. See Mike Cohn’s excellent pdf on how to fail with agile.

Prevention better than cure

Before you get all excited about the wonder cure to waterfall madness ask yourself:

• Is an agile methodology a good fit for your software development team? Or the bigger question: Is agile the right thing to do for your business?
• Are you sufficiently fed up with waterfall to try something radically different?
• How susceptible is your team/organization to change (managers, developers, testers, everyone!) ?

Stick it through

If your agile implementation is hurting more than delivering the expected result i advise:

• Call in an agile expert to coach your agile implementation
• Seek to get the most out of retrospective meetings. this means looking deeply at the root causes not just the manifestation of the problem(s)
• Talk to other agile practitioners (preferably “ri” level). You’re not alone
• Seek to understand the principles behind the rules and don’t give up until you do!

Disclaimer: I’m no agile expert. At best I may rate myself at “Ha” level. So if u find an odd statement or two in this post please send me some feedback. I would hate to mislead a “shu”

Try-catch suppress?

PHP never ceases to amaze me. Just the other day a colleague discovered that you can suppress error messages reported by file_get_contents() using the try-catch statement. That should’ve raised an eyebrow. In case you missed it here’s the sample code run from php’s interactive console:

Sample code, nothing odd (yet)


qbook:mobilesns quinton$ php -a
Interactive mode enabled

<?php

echo file_get_contents('bleh');

Warning: file_get_contents(bleh): failed to open stream: No such file or directory in /Library/WebServer/workspaces/default/mobilesns/- on line 3

Call Stack:
19.3675 673612 1. {main}() /Library/WebServer/workspaces/default/mobilesns/-:0
19.3914 673676 2. file_get_contents('bleh') /Library/WebServer/workspaces/default/mobilesns/-:3

Dump $_SERVER
Dump $_GET
Dump $_POST

Predictably, the code snippet above confirms that php will report an error if you attempt to open a file that does not exist. You may have noticed, my php installation has some extra debug output because I’m using the XDebug extension. I recommend it. Highly.

This is where it gets weird

Again, from the console. Here we place the same call to read a non-existent file within a try-catch statement.


qbook:mobilesns quinton$ php -a
Interactive mode enabled

<?php

try
{

echo file_get_contents('bleh');

}
catch (Exception $e)
{

echo "Exception is caught";

}

This time around the error is not reported by php nor is the “catch” executed proving that the error message is suppressed. Is it a bug or a side-effect? Anyone have any thoughts on this? “Its not a bug its a feature!” ~ comes to mind

My php version, in case you’re wondering


qbook:mobilesns quinton$ php -v
PHP 5.2.5 (cli) (built: May 27 2008 11:45:48)
Copyright (c) 1997-2007 The PHP Group
Zend Engine v2.2.0, Copyright (c) 1998-2007 Zend Technologies
with Xdebug v2.0.4, Copyright (c) 2002-2008, by Derick Rethans
with Zend Debugger v5.2.14, Copyright (c) 1999-2008, by Zend Technologies


session-clustering with memcache

You need php session-clustering and you need it done yesterday. The project is at risk. The suits are breathing down your neck and monitoring your every tweet. Memcache to the rescue

Installing memcached

Not installed? Get it here. Alternatively, install via a *nix package manager eg. apt, yum, rpm etc. Start your memcache servers:


/usr/local/bin/memcached -u root -d -m 1024 -l 0.0.0.0 -p 11211


Add one more to the pool: (this will make sense later. promise)


/usr/local/bin/memcached -u root -d -m 1024 -l 0.0.0.0 -p 11212


Note: the port number differs for the second memcached daemon.

Installing memcache extension with “session feature”

PHP “talks” to a memcache server via the memcache extension but you already knew that. Something less known is that when installing this extension you can enable/disable the extension’s session.save_handler feature.


qbook:Locate quinton$ sudo pecl install memcache
Password:
downloading memcache-2.2.5.tgz ...
Starting to download memcache-2.2.5.tgz (35,981 bytes)
....done: 35,981 bytes
11 source files, building
running: phpize
Configuring for:
PHP Api Version: 20041225
Zend Module Api No: 20060613
Zend Extension Api No: 220060519
1. Enable memcache session handler support? : yes

Choose “yes”. You can now configure php to use memcache as its session store.

Configuring the memcache session store

ext/memcache introduces new directives to the php.ini. In this post I’ll use ini_set() within a php script so you can just copy-and-paste to try it out yourself. In your production environment you’ll prolly want these directives inside your php.ini and/or apache vhost. You decide.


<?php

// firstly, override the default session save_handler like this
ini_set('session.save_handler', "memcache"); // PHP_INI_ALL Supported since memcache 2.1.2

// now tell php where to store the sessions. two memcache servers specified here. more bout that later
ini_set('session.save_path', "tcp://localhost:11211, tcp://localhost:11212"); // PHP_INI_ALL Supported since memcache 2.1.2

// getting interesting. if primary server is down talk to the others in the server pool
ini_set('memcache.allow_failover', "1"); // PHP_INI_ALL Available since memcache 2.0.2.

// more tweaking stuff follows here

// may want to adjust this
ini_set('memcache.max_failover_attempts', "20"); // PHP_INI_ALL Available since memcache 2.1.0.

ini_set('memcache.default_port', "11211"); // PHP_INI_ALL Available since memcache 2.0.2.

ini_set('memcache.chunk_size', "8192"); // PHP_INI_ALL Available since memcache 2.0.2.

ini_set('memcache.hash_strategy', "standard"); // PHP_INI_ALL Available since memcache 2.2.0.

ini_set('memcache.hash_function', "crc32"); // PHP_INI_ALL Available since memcache 2.2.0.

?>


A crude test

This code can be appended to the php snippet from above


<?php

session_start();

echo "Session save_handler is: ".ini_get("session.save_handler")."
";
echo "Session save_path is: ".ini_get("session.save_path")."
";

if(isset($_SESSION['bleh']))
{
echo "Bleh is already set: ".$_SESSION['bleh']."
";
}
else
{
$_SESSION['bleh'] = 'bwahahha';
echo 'Bleh has been set with: '.$_SESSION['bleh']."
";
}

?>

Where’s the clustering you ask

Well, truth be told there isn’t any. What we have so far is a memcache pool consisting of 2 servers. PHP is configured to write to the pool. PHP reads/writes to the server pool in order specified by “session.save_path” ini directive. For reads PHP will request a cache object by key from the pool. Because “failover” is enabled PHP will query the pool of memcache servers one-by-one until the request is fulfilled. Every silver lining has a cloud.

Fragmentation

In the event of a memcache server crash, fragmentation will occur. Therefore the pool will have an identical cache and its exceptionally hard to tell which keys each memcache server is storing. When a memcache daemon crashes it loses all data. Memory, by nature, is volatile. You have been warned.

Replication

This method is most commonly refered to as client-side replication BUT is better described as redundancy. There is a method of achieving server-side replication (think mysql replication). AFAIK repcached is the only available patch of its kind. I would like to try it but haven’t had the opportunity. Its a long story.

Summary

Memcache-based session save_handler is a quick-win but its NOT scalable. As your server farm grows the cost of maintaining and managing this solution is complexity. Also, consider the implications of managing a php.ini per web server and the repercussions of the failover feature. Nasty. However, there is good news. A libmemcached-based extension is on the way.

Getting real

If you have more time to spare take the safer option ie. storing sessions in a database. You could write a custom session save_handler or just use Zend framework’s Zend_Session_SaveHandler_DbTable. What could be easier?

Test-drive: MySQL Workbench

Workbench?

MySQL Workbench is a cross-platform, visual database design tool developed by MySQL. It is the highly anticipated successor application of the DBDesigner4 project. MySQL Workbench will be available as a native GUI tool on Window, Linux and OS X.

Some history

MySQL Workbench has been in the making for a while. Building on from DBDesigner 4 I’m sure there’s a community of anxious MySQL developers who can’t wait for a free and quality ERD-modelling (and more) tool for MySQL. The mysql command-line client is brilliant. I love it and will always use it BUT there’s no substitute for managing entity relationships visually.

I’ve been waiting on MySQL Workbench for ages (more specifically a Mac OS X release). On the day the windows version was released I soon installed it in my Windows XP virtual machine. It works really well but realistically its too painful to use via VirtualBox. Could be a breeze with VMWare Fusion. Anyone tried this?

It really is Alpha

MySQL Workbench 5.1.7 is an alpha release for Mac OS X and Linux which prolly explains why the dmg is only 7 megabytes. Features are missing!

And lift-off

Installation was easy. Typical drag-and-drop to Applications folder installation. First launch was all over in about 10 seconds. Workbench crashed when I tried File -> Open Recent menu item. Strange. Evidence follows:

Workbench Crash

And lift-off

On second launch I avoided “Open Recent” menu item. I tried the reverse engineer from .SQL script feature. Workbench smartly recreated a schema from an old project schema.sql I had lying around. I couldn’t edit any of the tables. Workbench complained about a “missing editor”. Maybe it wants to integrate with MySQL Query Browser or something?

Workbench reverse engineer script example

ERD’ing

Getting around an apparently buggy UI I did get to link some tables with many-to-one relationships. The experience was positively engaging. Workbench is already a vast improvement on the kludgy and buggy DBDesigner. Some screenshots to follow:

Workbench many-to-one example

And just to verify that the “Open Recent” crash wasn’t an anomaly I tried it again. Kaboom. A rather strange bug indeed.

Conclusion

If you’re going to use Workbench on real projects I advise practicing the “save every 5 seconds” rule. It is but only an Alpha but a very promising one at that. Regardless. I’m excited. I can’t wait for the first stable release. To the MySQL Workbench (and entire team at Sun MySQL) thanks for great tools and a great database. Love it. Bugs and all.

Using MySQL 5.1? You’ve been warned

The other day I got really excited about MySQL 5.1. Finally, I can use prepared statements and the query_cache at the same time! And then I stumbled upon Monty’s blog post.

In case you missed it. A stern warning from Michael Widenius (Founder and original developer of MySQL).

That’s alot of bugs. Its not the volume of bugs thats alarming but the seriousness attached to them. Many are “crashing” bugs! Depending on what you use MySQL for they may or may not matter. Depending on how much data you have that bug list could leave you feeling rather uncomfortable about MySQL 5.1.

Three months has gone by since MySQL 5.1 was announced “Generally Available”. According to mysql performance blog there are some exceptional MySQL users out there. Some on the bleeding edge using the 6.0 releases already.

Is this all a rush to become “Enterprise Ready”? Is it Sun’s fault? Has MySQL lowered its quality standards?

Who knows. I’m keeping my options open (postgres?). Some ex-MySQLers are busy elsewhere. Hrmm. Maybe the relational database is doomed after all.

Making an iso in mac os x leopard

The other day I desperately wanted/needed to install Ubuntu 8.04 Server as a virtual machine using VirtualBox. I could either install Ubuntu directly from the installation disc or install from a mounted ISO. I figured an ISO would be faster and more reliable. I looked all over Mac’s applications folder for a GUI utility but the for the life of me couldn’t find one to create an iso. I really thought Disk Utility would have that feature, it doesn’t right?

In *nix when the GUI fails you can always go old skool.

Open Terminal


qbook:~ quinton$ drutil status
Vendor Product Rev
MATSHITA CD-RW CW-8221 GA0K

Type: CD-ROM Name: /dev/disk3
Sessions: 1 Tracks: 1
Overwritable: 00:00:00 blocks: 0 / 0.00MB / 0.00MiB
Space Free: 00:00:00 blocks: 0 / 0.00MB / 0.00MiB
Space Used: 59:42:17 blocks: 268667 / 550.23MB / 524.74MiB
Writability:


Assuming you’ve already inserted the disc for copying. This command tells you where its been mounted.

Unmount the disk to be iso’d

qbook:~ quinton$ diskutil unmountDisk /dev/disk3
Unmount of all volumes on disk3 was successful


Using the location derived from the previous command. In my case the CD-ROM is /dev/disk3. Yours may differ

Make the iso


qbook:~ quinton$ dd if=/dev/disk3s0 of=ubuntu_8.04_lts_server.iso bs=2048
268667+0 records in
268667+0 records out
550230016 bytes transferred in 308.685634 secs (1782493 bytes/sec)


The real magic happens here. If you’re curious about progress you can watch the size of the newly created iso file “grow” in size using Finder or another terminal tab/window. Shouldn’t take longer than 10mins or so (for a 700mb cd)

Mount the ISO


qbook:~ quinton$ hdid ubuntu_8.04_lts_server.iso
/dev/disk1 /Volumes/Ubuntu-Server 8.


And to confirm the ISO is working. This command will mount the ISO as a readable volume. Should be viewable in Finder alongside all your other mounted devices.

VirtualBox’ing Ubuntu Linux

If you’re expecting a long hack about how to get it working sorry you won’t find it here. Ubuntu just works in VirtualBox except for one not so obvious error message on first boot.

I couldn’t get a screenshot for some reason but the error message you will get on first boot of any ubuntu distro (desktop or server) in VirtualBox is:


Starting up ...
This kernel requires the following features not present on the CPU:
0:6
Unable o boot - please use a kernel appropriate for your CPU.


Huh? Yeah me too. I’ve encountered this issue before I just forgot the fix so I had to google it (again). The fix is:

1. Power-off or shutdown the running virtual machine instance
2. Select the “Settings” menu option from the context-menu on your VM instance (mine is called hardy lts in the screenshot)
3. Then from the General tab pane select “Advanced” tab pane
4. Enable PAE/NX

That’s it! You now have a useable ubuntu virtual machine running in OS X. yay. This is listed as a “Known Issue” on the ubuntu wiki

Disclaimer: Steps may vary on your guest OS. eg. it might require 50+ clicks and a cold boot when using winblows

php4 and php5 certification

At the beginning of 2009 I found myself a bit idle. A new year in a new company always starts out a bit slow. I decided that the best use of this abundant time would be to get PHP5 certified.

Not the first certification

Back in early 2005 the Zend certification first became available for PHP4. At that time I had roughly 2 years php experience. No South Africans were certified at that time. Barely 100 certified engineers worldwide. What a great achievement if I could join the elite few.

PHP4 certification process

I signed into Zend.com and ordered the certification voucher, exam study guide and practice exam book. Zend were very accomodating and offered a generous discount on pricing.

The php4 certification guide provided adequate preparation for the exam. Along with that I gained a deeper understanding learning many things along the way. And I’m grateful for that.

The practice book is excellent. One could simulate at least 5 exams from the pool of questions in the book. It also provides the explanations to the answers which again was a real plus. Once I was ready I booked the exam with Pearson Vue at a nearby testing centre.

From purchasing the certification goods all the way to writing the exam it was a smooth experience and most worthwhile.

Fast forward to 2009

PHP5 certification process was a little different. Not nearly as smooth. This time around I ordered the pdf version of the certification study guide and instead of a practice exam book Zend now offers mock tests thru Vulcan.

Shortcomings in the study guide

PHP5 study guide lacks the depth offered by the php4 study guide. Perhaps its because PHP5 is a “bigger” language with additions such as MySQLi, PDO, Tidy, SOAP and all new XML extensions. Not forgetting the complete rework of the OOP model. Did I miss anything?

Vulcan needs work

To phparch and Zend’s credit they have devised a smart online mock testing platform but it needs refinement.

• I had a vulcan token which was valid for 10 mock tests but a few tests later the token become invalid.
• Questions were very poorly phrased. I couldn’t tell if they were trick questions or just a result of poor english.
• Vulcan didn’t offer explanations for the questions that you got wrong. It didn’t even tell you which questions you got wrong. Boo

Exam time

Exam format is 70 multiple choice questions in 90 minutes. Some code samples to interpret, some theory questions, some function names to type in but nothing too hardcore. The real exam also had some oddly phrased questions which often required a double-take. No sweat. I finished 40 minutes early and got my test results immediately. Pass. Yay!

In the end I missed my goal of becoming the first South African to be zend certified in 2005 and I also missed becoming the first php5 certified engineer. However, at time of writing I am the only South African to hold both php4 and php5 certifications. w00t!

Thats an achievement but that isn’t my motivation for getting certified. I revel in proving to myself that I’m a competent and capable PHPer. Things grow, change and evolve so quickly one must always make room for learning.