Dave Marshall's Software Blog

Silex Route Helpers for a Cleaner Architecture

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Tue, 27 Nov 2012 10:00:00 nil

In a previous post, we discussed creating your Silex controllers as Services, in which we created a nice, clean, easily tested controller, with few dependencies and collaborators. But I cheated a little, by returning a JSON response. Supposing we want to render some HTML, do we want to inject the template engine in to the controller? Should the controller be responsible for knowing how to render the template? I'm not sure, but if I can have it not do it with minimal fuss, I think I'd rather it not. The full stack framework has the @Template annotation, which allows developers to assign a template to a controller and then simply return an array. If they can do it in the full stack framework, we can do it in Silex.

This example is a little contrived, as the controller we end up with actually does nothing, so much so, that I've not even printed the method here. A better example would have the controller performing some sort of application specific logic, such as emailing the author, or updating some statistics, or whatever else you can think of.

``` php <?php

/* * What we're aiming for / $app->get("/posts/{post}", "posts.controller:viewAction")

->convert("post", $app['findOr404Factory']("Demo\Entity\Post"))
->template("post/edit.html.twig");

```

FindOr404Factory

This is really simple, if you're using doctrine, it simply creates closures for us that attempt to find an instance of the class we provide with the Id of the first argument, pretty much like the @ParamConverter annotation, without as much flexibility.

``` php <?php

$app['findOr404Factory'] = $app->protect(function($type, $message = null) use ($app) {

if (null === $message) {
    $lastNamespace = strrpos($type, "\\");

    if ($lastNamespace !== false) {
        $message = substr($type, strrpos($type, "\\") + 1) . " Not Found";
    } else {
        $message = "$type Not Found";
    }
}

return function($id) use ($app, $type, $message) {

    $obj = $app['em']->getRepository($type)->find($id);

    if (null === $obj) {
        return $app->abort(404, $message);
    }

    return $obj;
};

});

```

Custom Route Class

In order to have our nice template helper, we need to tell Silex to use a custom Route Class. This is pretty easy, but needs to be done before declaring any routes.

``` php <?php

class CustomRoute extends Route {

public function template($path)
{
    $this->setOption('_template', $path);
    return $this;
}

}

/* ... /

$app['route_class'] = 'CustomRoute';

```

TemplateRenderingListener

Out of the box, Silex comes with a listener that will attempt to convert anything you return from a controller (except for a Response object) to a string, and create a new Response with it. We need one of these, that will take an array and render a template if the route has one.

``` php <?php

class TemplateRenderingListener implements EventSubscriberInterface {

protected $app;

public function __construct(Application $app)
{
    $this->app = $app;
}

public function onKernelView(GetResponseForControllerResultEvent $event)
{
    $response = $event->getControllerResult();
    if (!is_array($response)) {
        return;
    }

    $request = $event->getRequest();
    $routeName = $request->attributes->get('_route');
    if (!$route = $this->app['routes']->get($routeName)) {
        return;
    }

    if (!$template = $route->getOption('_template')) {
        return;
    }

    $output = $this->app['twig']->render($template, $response);
    $event->setResponse(new Response($output));
}

public static function getSubscribedEvents()
{
    return array(
        KernelEvents::VIEW => array('onKernelView', -10),
    );
}

}

/* ... /

$app['dispatcher']->addSubscriber(new TemplateRenderingListener($app));

```

Job's a good'un. The code can be seen together in this gist, I haven't got a full working example I can publish at this time, but I can assure you it works! Hopefully you can see from this post how flexible Silex is, but also how much of a great reference the full stack framework is for whenever you want to do something new with your Silex boilerplate.

What we've ended up with is a PostController::viewAction method that doesn't care where the Post comes from and doesn't care how it's presented. All it care's about is the fact that the Post is being viewed, and what behaviour it should apply to the application domain, based on that fact. This soon goes to pot when you want to start redirecting to generated urls, persisting to the database etc but hey, one step at a time. Even then, it wouldn't take much to add a route helper that listens for a true boolean response and redirects to a given url. Same goes for a helper that would automatically flush Doctrine's Unit Of Work to the database, I'd be a little concious of doing that, but you get the idea.

One thing I would like to explore is applying security to the routes and their parameters, much like you can with the JMSSecurityExtraBundle, but that's for another day. I think the convert methods are applied literally last thing before executing the controller, so that might prove tricky.

I also should write about why I use Silex instead of the full stack framework...

Coding DCI in PHP

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Fri, 16 Nov 2012 11:59:00 nil

I've been trying to further my knowledge a lot recently and it has taken me out of my comfort zone quite often, but I think it's a great way to learn. One of the things I came across a while back was Data, Context and Interaction (DCI), but it wasn't until I heard Jim Gay talking about it on the /dev/hell podcast that I thought about taking a closer look, particularly back in my comfort zone of the PHP ecosystem. Further reading took me to The Right Way to Code DCI in Ruby, which I used to drive my PHP DCI experiment.

I wont go in to DCI theory too much, but there are lot's of articles around the web, if you fancy some deeper reading, Jim Gay has a book (currently beta) available at [clean-ruby.com](http://clean-ruby.com). I've not read it yet, but people are saying great things about it.

User Stories

Taken straight from the aforementioned post, our story is "As a user, I want to add a book to my cart". We can write a feature for that:

``` gherkin Feature: User adds a book to their cart

In order to purchase a book
As a user
I need to be able to add the book to my cart

Scenario: User adds a book to their cart
    Given I am a logged in user
    And a book exists
    And I am looking at the book's page
    When I click the add book to cart button
    Then the book should be in my cart

```

The "Roles"

PHP is not Ruby. PHP is defintely not Ruby. We've not actually written any code and we've hit our first sticking point. Ruby allows dynamic method injection and PHP does not. As such, I'm going to wrap our data with a role and delegate to it. This doesn't strictly adhere to the OOP utopia that DCI requires, but it is in "the spirit of DCI" (see comments in this post). We could come up with some magic solution, such as a trait that allows binding of closures to the data object, but I'm quite happy with the cleanness of this solution, without introducing a lot of userland magic.

Specs first, I'm using my own toy framework for this, I haven't really had chance to checkout phpspec2 and get fully up to speed on it.

``` php <?php

use BookShop\Entity\User; use BookShop\Entity\Book; use BookShop\Role\Customer;

describe("BookShop\Role\Customer", function() {

beforeEach(function() {
    $user = new User();
    $this->customer = new Customer($user);
});

describe("#addToCart", function() {
    it("puts the book in the cart", function() {
        $book = new Book;
        $this->customer->addToCart($book);
        assertThat($this->customer->getCart()->hasBook($book), true);
    });
});

});

```

To hide away the actual delegation, I'm using a simple trait, you can see it in the repo;

``` php <?php

namespace BookShop\Role;

use BookShop\Entity\Book; use BookShop\Entity\User; use BookShop\Util\Delegator;

class Customer {

use Delegator;

public function __construct(User $user)
{
    $this->delegateTo($user);
}

public function addToCart(Book $book)
{
    $this->getCart()->addBook($book);
}

} ```

Note: I've used a fairly loose interface here, I think ideally the argument passed to the constructor would need to be something like a CartOwner, allowing other types of object (anything that has a getCart method) to become a Customer. Maybe we'll have Visitor objects, for people who don't want to become a registered user etc.

The "Context"

I hit another stumbling block here. Because we are wrapping our data with our Customer role, we create a hard dependency on that class, making it difficult to mock. There are ways around this, I've seen some implemetations that pass the Role in to the Context, but I think that definitely goes against the "spirit of DCI", another way would be to optionally inject a factory into the context, specifically for testing, but seeing as we're dealing with a fairly simple interaction, I'll just use concrete instances.

``` php <?php

use BookShop\Context\AddToCartContext; use BookShop\Entity\User; use BookShop\Entity\Book;

describe("BookShop\Context\AddToCartContext", function() {

beforeEach(function() {
    $this->user = new User();
    $this->book = new Book();
    $this->context = new AddToCartContext($this->user, $this->book);
});

it("creates a customer", function() {
    assertThat($this->context->getCustomer(), anInstanceOf("BookShop\Role\Customer"));
});

it("adds the book to the customers cart", function() {
    $this->context->execute();
    assertThat($this->context->getCustomer()->getCart()->contains($this->book), true);
});

});

```

Another slight deviation here, we can't have a class method and an instance method called by the same name in PHP, so I just skipped the class method. Here's the implementation:

``` php <?php

namespace BookShop\Context;

use BookShop\Entity\User; use BookShop\Entity\Book; use BookShop\Role\Customer;

class AddToCartContext {

protected $customer;
protected $book;

public function __construct(User $user, Book $book)
{
    $this->customer = new Customer($user);
    $this->book = $book;
}

public function execute()
{
    $this->customer->addToCart($this->book);
}

public function getCustomer()
{
    return $this->customer;
}

} ```

The "Data"

I won't go in to that here, but our data classes are just POPOs and I use doctrine to persist them to the database. The relational side of things isn't perfect (you can only have one copy of a book in your cart), but that's not the focus of this. Admittedly, if I had a more robust data model (User -> Cart -> CartItem -> Book), to make doctrine fit might mean the Customer#addToCart method gets a little more complicated, though maybe that would be extra justification for using DCI and putting that logic in a Role.

``` php <?php

namespace BookShop\Entity;

class Book {

protected $name;

/**
 * Getter/Setters etc
 */

} ```

Fitting Into Silex

I'm a big fanboy of Silex, so here's how the context gets used in my controller:

``` php

$app->post("/cart", function(Book $book) use ($app) {

(new BookShop\Context\AddToCartContext($app->user(), $book))->execute();
$app['em']->flush();
return $app->redirect($app->path("book_view", array("book" => $book->getId())));

})->convert("book", / fetch book using id in request /);

```

We create the context and execute it and then flush changes to our datastore

Final Words

I quite enjoyed investigating DCI and in conclusion, despite PHP's implementation shortcomings, I think pursuing the "spirit of DCI" in PHP is a viable pursuit. It's quite possible that PHP's limitations in the architecture side may prove beneficial, in that DCI may be communicated more as the paradigm, separating the domain, the use cases and the roles, rather than just the code injection.

Silex Controllers as Services

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Wed, 03 Oct 2012 01:04:01 nil

**UPDATE**: The functionality described below is now baked in to Silex, checkout the [ServiceControllerServiceProvider](http://silex.sensiolabs.org/doc/providers/service_controller.html).

**TL;DR** Use a custom controller resolver to load controllers from the service container - [code][code]

I'll start by saying this is rarely the right thing to do with Silex. It's meant to be a micro framework, so if you're going to be building something that requires organising controllers in separate files or modules, maybe the full stack symfony framework would be a better choice. However, if you're like me and have gradually evolved a legacy application to be a Silex application, you may be in a situation where you want to try and keep things organised.

There's currently a pull request in the queue for Silex that adds a cookbook entry for [using controller classes][Cookbook], but I wanted to take it a step further and have my controllers as services, much like what's possible with the full symfony framework (See Richard Miller's post for further reading).

Things are best explained with an example, so let's start with this simple silex app:

``` php <?php

use Silex\Application; use Demo\Repository\PostRepository;

$app = new Application;

$app['posts.repository'] = $app->share(function() {

return new PostRepository;

});

$app->get('/posts.json', function() use ($app) {

return $app->json($app['posts.repository']->findAll());

});

```

First we need to tell Silex how we're going to map routes to our controllers. We do this by overriding the controller resolver with our own custom resolver. We can take the existing controller resolver and override the createController method. If we encounter a single colon in the controller name, we look to see if the string before the colon is a service, if it is, then we return a simple PHP callback array.

``` php <?php

namespace Demo\Controller;

use Silex\ControllerResolver as BaseControllerResolver;

class ControllerResolver extends BaseControllerResolver {

protected function createController($controller)
{
    if (false !== strpos($controller, '::')) {
        return parent::createController($controller);
    }

    if (false === strpos($controller, ':')) {
        throw new \LogicException(sprintf('Unable to parse the controller name "%s".', $controller));
    }

    list($service, $method) = explode(':', $controller, 2);

    if (!isset($this->app[$service])) {
        throw new \InvalidArgumentException(sprintf('Service "%s" does not exist.', $controller));
    }

    return array($this->app[$service], $method);
}

}

```

We simply then override Silex's default resolver.

``` php <?php $app['resolver'] = $app->share(function () use ($app) {

return new Demo\Controller\ControllerResolver($app, $app['logger']);

}); ```

We can now convert our controller to be a class, add it to Silex as a service ready to be instantiated, complete with dependencies, when needed:

``` php <?php

namespace Demo\Controller;

use Demo\Repository\PostRepository; use Symfony\Component\HttpFoundation\JsonResponse;

class PostController {

protected $repo;

public function __construct(PostRepository $repo)
{
    $this->repo = $repo;
}

public function indexJson()
{
    return new JsonResponse($this->repo->findAll());
}

}

```

``` php <?php $app['posts.controller'] = $app->share(function() use ($app) {

return new PostController($app['posts.repository']);

});

$app->get('/posts.json', "posts.controller:indexJson");

```

If I'm honest, the jury is still out on this technique for me. The benefits are that my controller code is a little more organised and I can actually practice spec level BDD with my controllers, rather than just functional tests, but sometimes I really just feel like throwing a closure in rather than knocking up a class. Maybe it might work for you, maybe not. Thanks for reading, full code example on github.

How I'm doing TDD with PHP

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Wed, 06 Jun 2012 09:12:12 nil

I've been watching the Destroy All Software back catalog over the last couple of months and it's really inspired me to up my TDD game. I'm still fairly new to TDD, I've written tests for a long time, but never really let it lead my development...

Outside-In or Middle-Out

I think I'm a middle-out kind of guy, I usually start with the business domain and try and decide what my API needs to offer. This means writing unit tests for some sort of Service class. From there, I'll branch out in to TDDing this service's collaborators.

Unit Tests

I use PHPUnit for my unit test needs, I think it's pretty much the defacto xUnit framework for PHP, I just did a quick search for SimpleTest and it's still hosted in SVN on SourceForge, enough said!

Granular

I try and write unit tests that are unit tests, not integration or functional tests. They take small (granular) pieces of code and test them.

Complete Isolation

I write my unit tests in almost complete isolation. It can get pretty messy, particular if you have a lot of 'glue' like services, bending several collaborators to their will. I make heavy use of dependency injection in the design of the code, and use lots of mock objects to isolate the SUT, then verifying a mixture of both behaviour and state, depending on how complex the SUT or it's collaborators are.

I've recently been putting in a lot of effort to relax the expectation constraints on the mock objects I use, so they're wired for expectations, but I don't verify them. This means I can test the behaviour of a method in individual tests if necessary, without having to repeat test code. I use mockery for this especially with it's shouldIgnoreMissing method, which essentially gets the mock to act like a Null Object. I'm hoping this will have the effect of making my tests easier to understand, in that you can see I'm only trying to verify the behaviour described by this test , and also I'm hoping it will lead to my tests being slightly less tied to the implementation.

Speed

Writing tests in isolation like this means my unit tests are fast. I don't hit any external services, I don't load any fixtures. My current test suite runs in less than 2 seconds.

Easy access

Gary from destroyallsoftware.com knows his way around vim. I had a look through his .vimrc and ended up copying loads of stuff from it. One of the most useful things I found is a few functions that will try and work out the test file for the current class you are editing and vice-versa. <leader>. will now switch between my production code and my unit tests. <leader>t will write the current file and run those tests (and will always remember the last run file, incase you switch back to production code or another file). The same mapping with uppercase T will run the tests and show PHPUnit's ascii code coverage report. This makes running tests so easy, there's no excuse not to. Checkout my .vimrc.

Fail first

I've been a lazy unit tester in the past, and probably wouldn't have made the effort to ensure a test can and will fail under the right circumstances, particularly when retro-fitting tests. With the nice easy access, mutating a test so that it will fail and running can easily be accomplished with less than a dozen keystrokes.

Fake it 'til you make it

Something else I've not done before, this involves writing a test, writing the minimum amount of code to make that test pass. Write another test, write the minimum amount of code to make both tests pass, and so on.

Craving RSpec's DSL

Watching those videos on destroyallsoftware.com makes me crave the nice DSL that comes with RSpec. Duck punching the should and should_not methods on to everything makes the expectations look swell, same goes for the stub method.

For example, here's the example spec file given on the rspec website for a bowling game

``` ruby

bowling_spec.rb

require 'bowling'

describe Bowling, "#score" do it "returns 0 for all gutter game" do

bowling = Bowling.new
20.times { bowling.hit(0) }
bowling.score.should eq(0)

end end ```

PHPSpec goes some way to trying to match it, but doesn't do enough to make me want to switch from phpunit. The same example from the PHPSpec website:

``` php class DescribeNewBowlingGame extends \PHPSpec\Context
{

private $_bowling = null;  

public function before()  
{  
    $this->_bowling = $this->spec(new Bowling);  
}  

public function itShouldScore0ForGutterGame()  
{  
    for ($i=1; $i<=20; $i++) {  
        // someone is really bad at bowling!  
        $this->_bowling->hit(0);  
    }  
    $this->_bowling->score->should->equal(0);  
}

} ```

That's before getting into stubs and mocks! There's not a real lot that PHPSpec can do, Ruby's syntax and flexibility plays straight in to rspec's hands. I don't actually like the idea of duck punching in production code, but it's use in a testing tool like this leads to an enjoyable experience.

I've also gone to great lengths over the last few years to make proper use of Dependency Injection or Inversion of Control in my code to really enable isolated unit testing, Ruby's ability to duck punch objects and in this case the so called 'singleton' object, makes it look pointless. Rubyists seem to write class methods everywhere, probably because they're so easy to mock and therefore easy to test as a collaborator, but also I think there's a lot to do with the libraries they use, like ActiveRecord. I'm sure there are other benefits of my approach (de-coupling is generally a good thing), but perhaps it would be nice to apply it a litte less liberally.

Integration Tests

Writing unit tests in the way I've mentioned above simply isn't enough for most types of system. Heavy use of mock objects can easily hide cracks in the itegration between different systems, these need testing. For these purposes, I use Behat. These tests tend to test most of the stack, with only a few external services, such as mail or third party APIs that are stubbed out somehow. As such, these tests are fairly slow, taking over a minute to run. At some point I intend to make a push and start testing some the of the UI interaction, which will require a javascript enabled browser for these tests, which will slow them down even further, despite recent breakthroughs with things like Zombie.js and PhantomJS.

Breaking the cucumber rules

I usually cuke it wrong and I don't write my features in the proper 'story BDD' style. My features aren't really acceptance tests, they have fairly declaritive setups and steps, and quite often involve mutliple actors, interacting as they would as part of a workflow. I make full use of the standard Mink DSL in order to rapidly develop features, rather than grafting my own DSL on top to create nicer looking feature files and a layer of abstraction that I probably wont use.

Continuous Integration

I'm a lone gunman in my current position, so continuous integration is downplayed a little, but I have Sismo configured to run composer, phpunit and then behat after each commit. It plays sounds on success/failure and does me for now. I did have Jenkins setup and pimped out to the max via Jenkins-PHP, but I found I wasn't getting much out of it.

That's all for now, will maybe follow up in a couple more months with more about my progress. I've recently picked up GOOS, I'm sure I'll learn how I'm doing it wrong when I get through that.

Why aren't you contributing to Open Source?

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Thu, 08 Mar 2012 12:38:18 nil

I've been publishing a bit of code recently and I'm loving it! My current job allows me to pick and choose the libraries and tools I use, and as such I've been spending plenty of time analysing them, making good use of them and where github's involved, giving back where I could. In the past I've submitted a couple of patches through bug trackers, but in the last couple of months, I've sent half a dozen, albeit minor pull requests to projects I use:

I've also started publishing more of my own work:

davedevelopment/housey - Early stages of a A/Bingo port to PHP
davedevelopment/stiphle - A simple PHP library for throttling things
davedevelopment/sismo-geckoboard-notifier
- a Sismo notifier for Geckoboard, written to (possibly) submit to Ibuilding's latest challenge
Audio notifier for Sismo (gist) - A really simple notifier that I use, along with Super Mario sounds effects :)

I think I've got past the point of caring too much about what other people might think about my work and standards like PSR0 and tools like composer have made it easier for me to create re-usable components that people may actually use. The two libraries I mentioned above published aren't watched and haven't been forked yet, but someone may find them useful one day! One of my other projects phpmig has seen some use and has had contributions from others.

The next step is to get my teeth in to something a bit more involved, I've got my eye on a particular issue in composer, but I'm need to speak to the core devs to get some guidelines on how to proceed, i.e. how much refactoring I can do without compromising the 1.0.0 release.

In conclusion, Github makes it so easy, why aren't you contributing?

How I'm designing a RESTful(ish) web service

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Thu, 16 Feb 2012 21:13:22 nil

Introduction and Disclaimer

This post is going to describe how I've ending up designing, what I consider to be a fairly RESTful web API. I'm far from being an expert, and this is definitely the closest thing to a RESTful API that I've ever created, so I'm not even experienced with REST APIs. So if I were you, I wouldn't take anything here as REST gospel. If you've got some criticisms, let me know, I'm all about the learning. There's also so much more to a RESTful architecture than what I describe in this blog post, this is simply what it took for me to find my way on to the path.

Until about 6 months ago, I'd always been sceptical of creating RESTful APIs, but I think I've had a few pennies drop since then that have made me fairly confident that I grasp the basics pretty well.

One of the biggest problems I've had was finding really good concreate examples, but recently the huddle-apis documentation popped up on my twitter timeline (via @bwaine, @dzuelke and @darrel_miller,) and I found it really useful.

Authentication

Something I wanted to clear up straight away was authentication. One of my biggests misconceptions about REST was the 'Stateless' constraint. I took that way too literally (or maybe I didn't and I now I'm just ignoring it). I'm now of the opinion that a session cookie is a valid way to authenticate a consumer for a REST API. That said, I wanted to go better than that, so I went and implemented WSSE authentication as described by the ATOM API. Now my application supports both, so the API can be consumed via XHR from the websites pages, and also by the more 'stateless' WSSE authenitcation headers. I've actually implemented this using Symfony's security component, which could be an entire blog post unto itself. In retrospect, I think I probably should have considered OAuth, but I always thought that was more about authorisation than authentication.

Richardson Maturity Model

The Richardson Maturity model is where I started with regards to setting goals for designing the actual web service. My basic knowledge of REST web services would have me meeting level 2 fairly easily, so I needed to brush up on the two things that would take me fully to level 3.

HTTP Verbs - PUT vs POST

Admittedly I had a rough idea of correct use of the HTTP verbs, but the penny didn't fully drop on the POST vs. PUT until I really started digging in to articles. My take on it is now, if you're use PUT, you'll always be sending a full representation of the resource you expect to find at the URI for that resource. A POST would generally be to create something where you wouldn't necessarily know the URI of the newly created resource or where you aren't sending a full representation of the resource.

POST /messages - Add this message resource to the /messages collection (create)
PUT /messages - Store this collection of messages at /messages
PUT /messages/123 - Store this message resource at /messages/123 (create or update)
POST /messages/123 - Store these attributes in the message resource at /messages/123

I don't think my interpretation would ring completely true with the purists, but it makes sense in my mind and it gave me the chance to move on to more important things...

HATEOAS and Defining Media Types

I'm aiming to hit level 3 of the model, which means conforming with the Hypermedia as the engine of application state (HATEOAS) guiding principle. I'm not overly convinced by the whole principle within the context of my requirements, but I thought I'd give it a shot.

The first step for is to document the media types we're going to accept and serve through the API.

I decided to solely support JSON as the data format I'll be providing to consumers. Every time I provide or consume XML, I hope it will be the last, so I won't be bothering with that. I've no real interest in HTML either, although it is nice to be able to browse APIs in a browser. With that in mind, I did a bit of reading about HAL in JSON, and liked most of what I found, so decided to take a similar approach.

I attended the PHPNW conference last year, and picked up some good points about media types from Ben Longden's talk, mainly:

Don't version the API, version the media types
Leave room for expansion and you might not have to version the media types

You see a lot of APIs with URIs like /1/messages, where 1 would be the API version. This goes against the REST grain, affectively describing a messages resource in version 1 of the API, whereby /2/messages would actually be the same resource, but in version 2 of the API. I'm hoping to never have to take that route, if my API changes that drastically, I'll introduce a new version of the media types that are affected. Thereby I'll be able to serve either the version 1 representation of the /messages resource, or the version 2 representation.

Leaving room for expansion is an easy one. Rather than:

``` javascript

{
    "to": ["dave@atstsolutions.co.uk"]
}

```

We'd do:

``` javascript

{
    "to": [{"address": "dave@atstsolutions.co.uk"}]
}

```

If we wanted to add more information to each recipient at some point, we could do so without affecting existing consumers of the API.

Example - API Resource (our API's entry point)

Now, one of the HATEOAS principles is that the consumer should be able to navigate to the resource they require from an original endpoint. I'm not really too sure how narrow or wide that can be, but for argument's sake, I'm going to have an API resource at the root URI of my API, with a media type of application/vnd.acme.api+json. This way consumers should be able to hit the root uri to start with and navigate to the resource they need. Personally, I find it very hard to imagine myself doing it, I'd be looking up the resource uri once and then hard-coding it, but maybe that's just me.

I'm currently documenting the media types on a single in my technical documentation, so as an example, the API resource would look something like this:

application/vnd.acme.api+json media type:

``` javascript

{
    "_links": {
        "self": {"href": "/", "type": "application/vnd.acme.api+json"},
        "messages": {"href": "/", "type": "application/vnd.acme.messages+json"}
    },
    "version": "1.0",
    "author": "Dave Marshall"
}

```

I got introduced to Behat when I worked at Sky Betting and Gaming, and it's been an invaluable tool for me in the new job, particularly as it's enabled me to quickly create automated tests for rickety old legacy php scripts, where unit testing is out of the question. It's further worked out as a nice tool to both test and, coupled with the media types documentation, document the API's usage as well as behaviour.

__[Cuking it Wrong](http://elabs.se/blog/15-you-re-cuking-it-wrong)__: I've been using behat to write *integration tests* for my API, rather than *acceptance tests*. They're written by a developer, for developers and speak REST over HTTP rather than the language of the domain. They're also fairly declaritive, I find it easier to write specific tests and cover edge cases. I think it goes against the grain of most people doing BDD with cucumber/behat, but it works for me.

Given our new endpoint, I can write a couple of simple scenarios covering the end point and authentication:

``` gherkin Feature: Acme example API

In order to make things happen
As an API consumer
I need to make API calls

Scenario: Authentic consumer checks API details
    Given the following API users exist:
        | Username    | Password    |
        | username123 | password123 |
    And I am API user "username123" with password "password123"
    And I send "wsse" authentication
    And I accept "application/vnd.acme.api+json"
    When I send a GET request to "/"
    Then the api response status code should be "200"

Scenario: Authentic consumer checks API details with incorrect password
    Given the following API users exist:
        | Username    | Password    |
        | username123 | password    |
    And I am API user "username123" with password "incorrectpassword"
    And I send "wsse" authentication
    And I accept "application/vnd.acme.api+json"
    When I send a GET request to "/"
    Then the api response status code should be "401"

Scenario: Non-Authentic consumer checks API details
    Given the following API users exist:
        | Username    | Password    |
        | username123 | password    |
    And I am API user "incorrectusername" with password "incorrectpassword"
    And I send "wsse" authentication
    And I accept "application/vnd.acme.api+json"
    When I send a GET request to "/"
    Then the api response status code should be "401"

```

Example - Messages Resource

As I'm sure you noticed, the API resource representation example above includes a link to a messages resource. Given that link, we can right some gherkin to test the messages resource API.

``` gherkin Feature: Messages Resource API

In order to manage messages
As an API consumer 
I need to make create, read, update and delete messages resources

Background:
    Given the following API users exist:
        | Username    | Password    |
        | username123 | password123 |
        | username456 | password456 |
    And I am API user "username123" with password "password123"
    And I send "wsse" authentication
    And I accept "application/vnd.acme.api+json"
    When I send a GET request to "/"
    Then the api response status code should be "200"
    And I remember the JSON response value at "_links/messages/href" as "messages_uri"

Scenario: Authentic user retrieves a collection of messages
    Given the following messages exist:
        | From        | To          | Subject        | Body           |
        | username456 | username123 | Test Message 1 | Blah Blah Blah |
        | username456 | username123 | Test Message 2 | Blah Blah Blah |
    And I accept "application/vnd.messages.api+json"
    When I send a GET request to "<messages_uri>"
    Then the api response status code should be "200"
    And the JSON response should have 2 "messages"
    And the JSON response value at "messages/1/_embedded/message/subject" should be "Test Message 2"

```

This scenario above uses a background to 'navigate' to the messages resource uri via the API resource described earlier, and then tests the messages resource API. Let's take a look at the application/vnd.acme.messages+json media type documentation.

``` javascript {

"_links": {
    "self": { 
        "href": "/messages", 
        "type": "application/vnd.acme.messages+json"
    },
    "send-message": { 
        "href": "/messages/outbox", 
        "type": "application/vnd.acme.message+json",
        "method": "POST"
    }
},
"messages": [
    {
        "_embedded": {
            "_links": {
                "self": {
                    "href": "/messages/1",
                    "type": "application/vnd.acme.message+json"
                }
            },
            "from": {
                "name": "username456"
            },
            "to": {
                "name": "username123"
            },
            "subject": "Test Message 1",
            "body": "Blah Blah Blah"
        }
    },
    {
        "_embedded": {
            "_links": {
                "self": {
                    "href": "/messages/2",
                    "type": "application/vnd.acme.message+json"
                }
            },
            "from": {
                "name": "username456"
            },
            "to": {
                "name": "username123"
            },
            "subject": "Test Message 2",
            "body": "Blah Blah Blah"
        }
    }
]

}

```

Whoa, a little verbose! This is definitely the downside to trying to stick with the HATEOAS principles, but you kind of get used to it. A few things to note in the example:

Links - we've got a send-message link, pretty self explanatory. I've added the type and the method as attributes, as it's one thing I actually found to be missing in HAL+JSON. Most of the time the HTTP method to use can be assumed, but everything else is so explicit I thought why not. I'd do the same in a HTML form, so why not in my JSON representation.
Embedded Message - Rather than just a list of URIs to the individual messages, I've embedded some of the message contents
The from and to attributes of the messages are both JSON objects. Right now they only have a name attribute, but somewhere down the line we might introduce a user resource, at which point we could add links at the very least, if not embedded information such as full name and other attributes a user may have.

Now we know where to find the hyperlink to send a message, we can write an integration test that will discover the URI and create a message

``` gherkin

Scenario: Authentic user sends a message 
    Given the following messages exist:
        | From        | To          | Subject        | Body           |
        | username456 | username123 | Test Message 1 | Blah Blah Blah |
        | username456 | username123 | Test Message 2 | Blah Blah Blah |
    When I send a GET request to "<messages_uri>"
    Then the api response status code should be "200"
    And I remember the JSON response value at "_links/send-message/href" as "send-message-uri"
    Given I accept "application/vnd.message.api+json"
    And I send "application/vnd.message.api+json"
    When I send a POST request to "<send-message-uri>" with the following:
        """
        {
            "from": {
                "name": "username123"
            },
            "to": {
                "name": "username456"
            },
            "subject": "Hello",
            "body": "Blah Blah Blah"
        }
        """
    Then the api response status code should be "201"
    # and check the response attributes etc.

```

And that's about it. I'm really happy with the progress I've made, but please feel free to tear in to my practices in the comments below. I'll be back with some more REST related posts, probably on my implementation with Silex and probably some more details on the custom Behat contexts I'm using.

Defending against Cache Stampedes

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Fri, 13 Jan 2012 00:59:12 nil

Bonnie, there's a stampede... in your tent!
Mitch Robbins - City Slickers(1991)

I've recently had a problem with a rather large operation (that could probably be optimised considerably, but nevermind), where by if the cached result of the operation expired, several web server threads would attempt the operation, causing some major headaches for our database and web servers. This is something I've come across before, and is commonly(?) known as a Cache Stampede. This is bad, this post describes the basics of what I've done to deal with it.

Basic Caching

Given a Big Fecking Operation, we wrap some simple code around it using a Zend_Cache instance to cache the result for an hour - ignore the use of globals and lack of otherwise good coding practices

<?php

function bfo9000() {
    sleep(5);
    return time();
}

function bfo9000_cached() {
    global $cache;
    $key = 'bfo9000_cache';
    $res = $cache->load($key);
    if (false === $res) {
        $res = bfo9000();
        $cache->save($res, $key, array(), 3600);
    }

    return $res;
}

Semaphores

Once we start to get stampedes, the first thing we want to do is try and limit the amount of threads/processes that are going to try and regenerate the result and store it in the cache again. We can do this using a Binary Semaphore.

Note: Zend cache isn't exactly ideal for this, as the combination of test and save are not atomic, using something like Memcached::add would be better if you really want to avoid two processes generating the cached result.

Our function will now be programmed, when getting a cache miss, to check to see if the semaphore exists, if so, will sleep for a little while before checking again. If it doesn't exists, it will create the semaphore itself, before populating the cache and releasing the semaphore.

<?php

function bfo9000_cached() {
    global $cache;
    $key = 'bfo9000_cache';
    $lockKey = 'bfo9000_cache_lock';
    $res = $cache->load($key);
    if (false === $res) {

        while (false === $res && $cache->test($lockKey)) {
            sleep(1);
            $res = $cache->load($key);
        }

        if (false === $res) {
            $cache->save(true, $lockKey, array(), 10);
            $res = bfo9000();
            $cache->save($res, $key, array(), 3600);
            $cache->remove($cacheKey);
        }
    }

    return $res;
}

All being well, we should have prevented the stampede, as several of the threads that require the result of the operation will wait while one (or maybe a few) do the actual ground work.

Shortening the window

We've now prevented the stampede from bringing the servers to their knees, but we're not necessarily providing the best experience for those consumers that simply wait while the other threads do their stuff. To improve this, we can manage the expiry of the cache ourselves and attempt to prime the cache before it fully expires. I'm not sure if there's a name for this technique, but it's mentioned on the Memcached FAQ.

UPDATE: Adam's pointed out that Varnish would call this a Grace Period

We need to store the actual expiry with the data in the cache, rather than relying on the cache software's expiry, luckily for my example, Zend_Cache does that for us. When we get a cache hit, we check to see if the cached result is due to expire soon. If it is, we immediately update the cached result with an expiry date much further in the future, and then set about updating the cache. This way, other processes will see the cached result as valid while our process updates the cache.

<?php

function bfo9000_cached() {
    global $cache;
    $key = 'bfo9000_cache';
    $lockKey = 'bfo9000_cache_lock';
    $res = $cache->load($key);
    if (false === $res) {

        while (false === $res && $cache->test($lockKey)) {
            sleep(1);
            $res = $cache->load($key);
        }

        if (false === $res) {
            $cache->save(true, $lockKey, array(), 30);
            $res = bfo9000();
            $cache->save($res, $key, array(), 3600);
            $cache->remove($cacheKey);
        }
    } else {
        $meta = $cache->getMetaDatas($key);
        if ($meta['expire'] > time() + 300 && !$cache->test($lockKey)) {
            $cache->touch($key, 3600);
            $cache->save(true, $lockKey, array(), 30);
            $res = bfo9000();
            $cache->save($res, $key, array(), 3600);
            $cache->remove($lockKey);
        }
    }

    return $res;
}

Note in the main else block, if the semaphore already exists, we leave generating a new cached result to the process that's doing so and simply return the result we already have.

Disclaimer

I wrote all of the code above right here in the post, please don't copy and paste it in to your projects and complain to me if it doesn't work :) I think I've got it just about right and it seems to work for me, let me know nicely of all the mistakes I've made in the comments below.

Handling Symfony2 Events Asynchronously

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Wed, 02 Nov 2011 22:15:46 nil

UPDATE 23/01/2012

As Sebastien pointed out in the comments, Symfony 2.1's emails are sent after the page via a kernel.terminate event, which is cool! I don't know if there's any specific configuration required, but Fabien tweeted about it. Rather than going forward with what I propose in this blog post, you could use the kernel.terminate event, to improve perceived performance for your users. Things would take place in process still, so taking up web server resources, but it's a good first step.

I love Symfony2's Event Dispatcher and find it's a really nice way of decoupling code, though admittedly it can decouple things a little too much. One thing I constantly find, is that the listeners I bind to events tend to be for sending notifications or other sorts of logging, which in an ideal world, would be handled asynchronously. That is, why should this user wait for their page response, just so your system can email the 10 people who want to know about what they did? Most people taking the leap to move these functions offline would create message queues for each of those individual listeners, or setup a new separate service bus, but I've had a look at attaching a message queue to the existing symfony architecture, by creating a simple decorator for the event dispatcher.

Example

You can get the full example on github, I'll just show snippets here as a walkthrough. I'm using Silex for the example, but the same principles apply to a full blown symfony application, or any app that's using the Event Dispatcher.

Here some code for a simple JSON API to add a comment to a blog post. Notice that the email is sent inline:

/**
 * Add a comment
 */
$app->post('/blog/{post}/comments', function(Application $app, array $post) {

    $id = !empty($post['comments']) ? max(array_keys($post['comments'])) + 1 : 1;
    $comment = json_decode($app['request']->getContent(), true);
    $comment['id'] = $id;
    $post['comments'][$id] = $comment;

    /**
     * Send an email to the author
     */
    mail($app['author'], 'New comment on ' . $post['title'], $comment['comment']);

    return new Response(
        json_encode($comment),
        201,
        array(
            'Content-type' => 'application/json',
            'Location' => "/blog/{$post['id']}/comment/{$id}",
        )
    );
})->convert('post', $postLookup);

Using Events

Our first improvement is to send that email via an event listener. First we add a listener:

$app['dispatcher']->addListener('blog.comments.new', function(Event $event) use ($app) {
    mail($app['author'], 'New comment on ' . $event->post['title'], $event->comment['comment']);
});

Then we alter the controller to fire an event rather than firing an email

/**
 * Fire and event rather than sending an email
 */
$event = new Event;
$event->post = $post;
$event->comment = $comment;
$app['dispatcher']->dispatch('blog.comments.new', $event);

This is a nice little improvement, we've decoupled sending the email from the main purpose of the endpoint which is adding the comment. The problem is, the sending of the email, which can be slow, still happens in the same synchronous process, which we'll hope to overcome in the next few steps.

Firehose Event Dispatcher

@glenathan pointed out that storing and switching on the async prefix wasn't necessary, so that's been removed, see the full changeset for details.

I implemented this as a decorator (I think, I'm rubbish with patterns), and it basically allows you to register listeners that will receive all events dispatched through a given event dispatcher. Checkout the source on github. In this example, we're going to use the firehose to refire the event with a different event name:

$app['base_dispatcher'] = $app['dispatcher'];
$app['dispatcher'] = $app->share(function($c) {
    $firehose = new FireHose($c['base_dispatcher']);
    return $firehose;
});

$app['dispatcher']->addFireHoseListener(function($eventName, Event $event) use ($app) {
    /**
     * Disable the firehose, then fire the event again but with the async
     * prefix
     */
    $app['dispatcher']->oneTimeDisableFireHose();
    $app['dispatcher']->dispatch('async.' .  $eventName, $event);
});

Now we change the name of the event for our email sending listener

$app['dispatcher']->addListener('async.blog.comments.new', function(Event $event) use ($app)
    mail($app['author'], 'New comment on ' . $event->post['title'], $event->comment['comment']);
});

At this point, we're not doing anything asynchronously, but we've given our app the ability to make a choice. Listeners can choose to listen on the regular event name, or choose the asynchronous event name, by which they should expect to be notified asynchronously.

Fire hosing on to a message queue

The next step is to change the firehose listener to one that will push the event on to a message/job queue or service bus of some description. In this example, I've used 0MQ in a pub/sub type setup, but you can insert anything you like in here. If you're interested in 0MQ, checkout Ian Barber's Slides.

$app['dispatcher']->addFireHoseListener(function($eventName, Event $event) use ($app) {
    /**
     * Dont bother with the silex or kernel events, in real life you'd probably 
     * want to switch this on class type of the event
     */ 
    if (0 !== strpos($eventName, 'kernel.') && 0 !== strpos($eventName, 'silex.')) { 

        $eventName = 'async.' . $eventName;

        if ($app['dispatcher.async']) {
            $app['dispatcher.queue.pub']->send(serialize(array($eventName, $event)));
        } else {
            // might be useful to have in process for dev
            $app['dispatcher']->oneTimeDisableFireHose();
            $app['dispatcher']->dispatch($eventName, $event);
        }
    }
});

$app['dispatcher.queue.pub.dsn'] = 'tcp://localhost:5567';
$app['dispatcher.queue.sub.dsn'] = 'tcp://localhost:5566';
$app['dispatcher.queue.pub'] = $app->share(function($c) {
    $ctx = new ZMQContext();
    $send = $ctx->getSocket(ZMQ::SOCKET_PUSH);
    $send->connect($c['dispatcher.queue.pub.dsn']);
    return $send;
});

source on github

There's an important point here, note the exclusion of the kernel and silex events, these events dont serialize. If you want to pop your Events on a queue and have them handled elsewhere, they need to be in a portable format. Events that implement the serializable interface would be a good start, but you might want to consider what other systems may want to get to the data and use a format like JSON.

Now to create a little daemon to pull things off the queue and refire the event like we did before:

$app = include __DIR__.'/app.php';
$ctx = new ZMQContext();
$sub = $ctx->getSocket(ZMQ::SOCKET_SUB);
$sub->setSockOpt(ZMQ::SOCKOPT_SUBSCRIBE, '');
$sub->connect($app['dispatcher.queue.sub.dsn']);
$poll = new ZMQPoll();
$poll->add($sub, ZMQ::POLL_IN);
$read = $wri = array();
while(true) {
    $ev = $poll->poll($read, $wri, 5000000);
    if ($ev > 0) {
        list($eventName, $event) = unserialize($sub->recv());
        echo "Refiring $eventName\n";
        $app['dispatcher']->oneTimeDisableFireHose();
        $app['dispatcher']->dispatch($eventName, $event);
    }
}

source on github

Last but not least, we need a 0MQ server to sit in the middle:

$ctx = new ZMQContext();
$pub = $ctx->getSocket(ZMQ::SOCKET_PUB);
$pub->bind('tcp://*:5566');
$pull = $ctx->getSocket(ZMQ::SOCKET_PULL);
$pull->bind('tcp://*:5567');

while(true) {
    $message = $pull->recv();
    $pub->send($message);
}

source on github

Putting it all together

I know it's a rather crude example, but what you end up with is a pretty simple way of turning synchronous events into asynchronous ones.

It would be nice to hear other people's thoughts on this, hit me up with some comments. I know it seems a bit messy, but I'm betting there's quite a few symfony sites wired up with events that could benefit from having a few taken offline?

Phpmig - Simple migrations for php

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Tue, 01 Nov 2011 00:15:46 nil

I've got a serious Not Invented Here syndrome problem when it comes to (database) migrations.

I've previously blogged about migrations with phing and dbdeploy and also porting ActiveRecord::Migrations to PHP, now here I am again blogging about yet another way of doing migrations in PHP projects. Only maybe this time it's different, maybe this time I've found a way I'm happy with...?

My requirements were:

Ability to run 'interleaved' migrations
Migrate/rollback
Not tied to any framework/library or particular way of configuring your app
Migrations must be able to execute code, plain SQL isn't enough - think data manipulation, or migrating XML files etc
Pretty console colours
Easy install

Phpmig is a simple migrations system that was written to be easily adopted regardless of the framework or libraries you are using. It requires a little bit of setting up, but if you know you should be using migrations, you're probably more than capable.

It's really simple

On it's own, it doesn't really do a lot. It runs migrations.

You have to tell it where your migrations are. You have to tell it where to record which migrations have been run. You do this by passing it something that implements the ArrayAccess interface, with at least two keys, phpmig.adapter and phpmig.migrations. The former needs to be an instance of Phpmig\Adapter\AdapterInterface, and allows phpmig to record and decide which migrations have been run. The latter is an array of migration files.

For this, I recommend using Pimple, a small service/dependency injection container, there's a version bundled under the Phpmig namespace, Phpmig\Pimple\Pimple. The nice thing is, the very same ArrayAccess object is accesible from your migration files. Phpmig doesn't provide you with a way of accessing your database, seeing as that's probably quite specific to your project, depending on the module you're working on, or the environment you're working on, you'll have to take care of it yourself.

Example Applicaiton

If you want to get stuck in straight away, checkout the README, otherwise read on for a simple usage example.

I'm going to use Silex as the base for my example app, for two reasons. Firstly it's really nice, second because it uses an instance of Pimple as a service container.

A little setup first:

$ cd ~/src
$ mkdir phpmig-example-app
$ cd phpmig-example-app
$ wget http://silex.sensiolabs.org/get/silex.phar
$ mkdir vendor
$ git submodule add https://github.com/doctrine/dbal vendor/doctrine-dbal
$ git submodule add https://github.com/doctrine/common vendor/doctrine-common

Now we can move on to writing our app. Rather than having what's essentially the bootstrap in the index file, I put everything in a file called app.php

<?php

// app.php

require_once __DIR__.'/silex.phar';

$app = new Silex\Application();

$app->get('/hello/{name}', function($name) use($app) {
    return 'Hello '.$app->escape($name);
});

$app->register(new Silex\Provider\DoctrineServiceProvider(), array(
    'db.options'            => array(
        'driver'    => 'pdo_sqlite',
        'path'      => __DIR__.'/app.db',
    ),
    'db.dbal.class_path'    => __DIR__.'/vendor/doctrine-dbal/lib',
    'db.common.class_path'  => __DIR__.'/vendor/doctrine-common/lib',
));

return $app;

The index file simply includes the app.php file and runs the app.

<?php 

// index.php

$app = include __DIR__.'/app.php';

$app->run();

That's our app built, so now we can phpmig it:

$ sudo pear channel-discover pear.atstsolutions.co.uk
$ sudo pear install atst/phpmig-alpha
$ cd ~/src/phpmig-example-app
$ phpmig init
+d ./migrations Place your migration files in here
+f ./phpmig.php Create services in here
$

Phpmig has created a phpmig bootstrap file for us, but it's very basic:

<?php

use \Phpmig\Adapter,
    \Phpmig\Pimple\Pimple;

$container = new Pimple();

$container['phpmig.adapter'] = $container->share(function() {
    // replace this with a better Phpmig\Adapter\AdapterInterface 
    return new Adapter\File\Flat(__DIR__ . DIRECTORY_SEPARATOR . 'migrations/.migrations.log');
});

$container['phpmig.migrations'] = function() {
    return glob(__DIR__ . DIRECTORY_SEPARATOR . 'migrations/*.php');
};

return $container;

We don't need to use phpmig's bundled version of pimple, and we've got ourselves a nice doctrine connection setup in our app, so we may as well use that to record the running of our migrations, rather than a flat file.

<?php

use \Phpmig\Adapter;

$container = include __DIR__.'/app.php';

$container['phpmig.adapter'] = $container->share(function($c) {
    return new Adapter\Doctrine\DBAL($c['db'], 'migrations');
});

$container['phpmig.migrations'] = function() {
    return glob(__DIR__ . DIRECTORY_SEPARATOR . 'migrations/*.php');
};

return $container;

We can see if things are working by running phpmig's status command:

$ phpmig status

 Status   Migration ID    Migration Name 
-----------------------------------------

We can now create our first migration. Phpmig will create a skeleton for us, but we have to give it a name and a place to put it

$ phpmig generate CreateFirstTable ./migrations
+f ./migrations/20111101000144_CreateFirstTable.php

The file it generates contains two empty methods, up and down.

<?php

use Phpmig\Migration\Migration;

class CreateFirstTable extends Migration
{
    /**
     * Do the migration
     */
    public function up()
    {

    }

    /**
     * Undo the migration
     */
    public function down()
    {

    }
}

As our service container has a db connection for us to use, we can use that to create (or drop) our table.

<?php

use Phpmig\Migration\Migration;

class CreateFirstTable extends Migration
{
    /**
     * Do the migration
     */
    public function up()
    {
        $c = $this->getContainer();
        $c['db']->query("
            CREATE TABLE `user` (
                `id` INTEGER,
                `name` TEXT,
                PRIMARY KEY (`id`)
            );
        ");
    }

    /**
     * Undo the migration
     */
    public function down()
    {
        $c = $this->getContainer();
        $c['db']->query("DROP TABLE IF EXISTS `user`;");
    }
}

All that's left now is to run the migration we've just created

$ phpmig status

 Status   Migration ID    Migration Name 
-----------------------------------------
   down  20111101000144  CreateFirstTable

$ phpmig migrate
 == 20111101000144 CreateFirstTable migrating
 == 20111101000144 CreateFirstTable migrated 0.0037s
$ phpmig status

 Status   Migration ID    Migration Name 
-----------------------------------------
     up  20111101000144  CreateFirstTable

$

Comments, Feedback and Patches

All are welcome, it's still in it's infancy, but I'm using it on a couple of projects already and I'm fairly happy with it. I've not created pear packages (or indeed run a pear server) before, so go easy on me if I've screwed anything on that side of things.

Rambling On

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Tue, 25 Oct 2011 00:11:12 nil

2010 and 2011 have been very big years for me. Towards the backend of 2010, my beautiful daughter was born and my then girlfriend and I got engaged. Since then, I landed a great new job at Sky Betting and Gaming as a Senior Software Engineer, and now I've moved on again to work for a small startup in the form of Childcare.co.uk. And now I've updated my blog!

Sky Betting and Gaming

I took the decision in May to move on from Spencer's, I still enjoyed the job but felt that having been there for 7 years, across 5 (or maybe even 6) positions, I'd finally become pretty limited in terms of capacity for growth, so when I got the opportunity to go to SkyBet, I took it. It did mean a one hour commute each way but I was going to be working with a high quality team on a big site and with plenty of scope for improvement and progression.

Things went really well there and I really enjoyed it. It was nice to leave the management responsibilities behind at Spencers and go to Sky and just get stuck in to the code, and at first I found the code base quite overwhelming, but I picked things up and spent most of my time working on a JSON-RPC web service for placing bets.

It also gave me my first chance to be fully immersed in agile methods, mainly following Scrum, but there was a definite hint of Scrum-ban creeping into the life cycle, particularly when we had a fixed deadline invoked by the management team.

Sky seem to take good care of their staff, the working environment, nature of the work and the benefits package where all on the money, it was just unfortunate for me that another opportunity arose so quickly. I think Sky are hiring at the minute, so it's worth checking their job listings if you're on the lookout for new opportunities.

Before my last week they were good enough to let me join the team in going to PHPNW, I think there were about 10 of the technology team in attendance, all funded by the company, so they've got that part right as well. As always I enjoyed the conference, some talks more than others, but I mainly enjoy the social side of things anyway.

Childcare.co.uk

I've worked on Childcare.co.uk as a freelancer in the past, and as the site has seen excellent growth in the two and a half years it's been running, I was approached by the owner about joining them full-time. I've been in the job a week now and I'm really enjoying it, working for a much smaller company means I can rapidly make decisions and move towards being truly agile (I pushed 3 releases today!). It also allows me to engage in more architectural type decisions when the opportunity arises, decisions that I didn't really have the capacity to make as a freelancer when I've worked with them in the past and the kind of decisions that were made by more senior staff at Sky. I also get to work from home full-time!

In terms of what I'll be doing, I've got a good mix of general site enhancements, new features and a whole new sister project. The company is on the verge of launching mobile apps as well, and while it's hardly a specialty of mine, I wouldn't mind having a look around at some of that stuff.

I'm definitely going to miss the team atmosphere they have at Sky, I seemed to get along well enough with everyone there, and working at home on my own will be a stark contrast. To counter that a little, I'm going to make more effort with the networking events that are not too far away from me, starting with PHP Leeds and maybe even Hull Digital. It'd be really cool to get to one of the HNLondon meetups, but that's a bit of a trek for me.

Blog

Apologies to anyone who subscribes to the feed and got a barrage of crappy old posts, I'm not entirely sure what I goosed up there, but I saw the damage it did to my Google Reader experience.

The blog's now running on Jekyll, a static site generator that powers github pages. It's really cool and means I can hack together blog posts in markdown, push them to a repo and have them deployed automatically. It took a bit of getting set up and I've lost all the comments from the old wordpress blog, but I couldn't be bother to try and import them into Disqus. The design was really nice and easy thanks to Twitter Bootstrap, much better than my previous 'programmer design' work you've seen up here.

I don't think the comments are working at the minute, probably because Disqus haven't got the DNS change through yet, but if they are and you fancy leaving me some feedback, it's always appreciated.

Traits in PHP 5.4 - HelloWorld with Logging Trait

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Fri, 05 Aug 2011 23:08:45 nil

I'm looking forward to having traits in PHP 5.4 :)

One of the 'traits' I find myself constantly adding to library files is optional logging of it's behaviour. The library class has it's own log method, that checks to see if the instance has had a logger injected and if so, logs the message. I see this as a perfect candidate for becoming a reusable trait, as I tend to have the same code copy/pasted throughout my library classes.

The problem is, according to the rfc, traits aren't supposed to have state/properties, which makes it difficult to have a DI setter method in a trait. I guess it provides further protection against clashes, but you could always make the property name very unique, though I suppose that not a particularly good practice. However, setting properties in the traits seems to work just fine, so here's a quick demo of adding logging capabilities to a class via traits in PHP 5.4. Needless to say, because this may be undesired behaviour and it's only an alpha release, I'm not banking on this working in the future.

<?php

/**
 * Logging trait
 */
trait Logging
{
    /**
     * @var Zend_Log
     */
    protected $_logger = null;

    /**
     * Set logger
     *
     * @param Zend_Log $logger
     * @return void
     */
    public function setLogger(Zend_Log $logger) 
    {
        $this->_logger = $logger;
    }

    /**
     * Log something
     *
     * @param string $level
     * @param string $msg
     * @return void
     */
    protected function log($level, $msg)
    {
        if ($this->_logger !== null) {
            $this->_logger->{$level}($msg);
        }
    }

}

/**
 * Hello World
 */
class HelloWorld
{
    use Logging;

    /**
     * Run the hello world
     */
    public function run()
    {
        $this->log("info", "running");
        echo "Hello World!\n";
        $this->log("info", "done");
    }
}

/**
 * Traits demo
 */
date_default_timezone_set('Europe/London');
require_once 'Zend/Log.php';

$logger = Zend_Log::factory(array(
    array(
        'writerName'   => 'Stream',
        'writerParams' => array(
            'stream'   => 'php://stdout',
        ),
    ),
));

$hw = new HelloWorld;
$hw->setLogger($logger);
$hw->run();

After downloading and compiling the latest alpha, I can run the code seeing the logging I wanted.

davem@marajade:~$ cd src
davem@marajade:src$ tar -zxvf ~/Downloads/php-5.4.0alpha3.tar.gz
davem@marajade:src$ cd php-5.4.0alpha3
davem@marajade:php-5.4.0alpha3$ ./configure --disable-all
davem@marajade:php-5.4.0alpha3$ make
davem@marajade:php-5.4.0alpha3$ cd ~/src/traits-demo
davem@marajade:traits-demo$ ~/src/php-5.4.0alpha3/sapi/cli/php run.php 
2011-08-05T23:33:15+01:00 INFO (6): running
Hello World!
2011-08-05T23:33:15+01:00 INFO (6): done
davem@marajade:traits-demo$

Copy install packages to new machine

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Tue, 19 Apr 2011 09:37:53 nil

Note to self as much as anything, taken from http://ubuntuforums.org/showthread.php?t=261366.

Old machine:

> sudo dpkg --get-selections > installed-software

New Machine:

> sudo dpkg --set-selections < installed-software
> sudo dselect

Asynchronous cache priming with progress bars

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Mon, 04 Apr 2011 08:09:13 nil

Bit of a long winded title, but I wanted to get it all in there.

EDIT: The long title broke my new design, so I shortened it :)

Why?

I have a (highly optimised) report that takes way too long to generate, up to around 30 seconds
Too many variables to prime caches for every possible combination
Jakob Nielson of usability fame says:
10 seconds is about the limit for keeping the user's attention focused on the dialogue. For longer delays, users will want to perform other tasks while waiting for the computer to finish, so they should be given feedback indicating when the computer expects to be done. Feedback during the delay is especially important if the response time is likely to be highly variable, since users will then not know what to expect.
Personally, I don't think the browsers inbuilt progress bar is enough feedback for todays web users

How?

First off, I configured the report to cache the generated data in memcached for a day. That's great, now if the user requests that particular report (with the exact same combination of filters etc), twice in a day, the second time will be really snappy. What I now need is a way of priming the cache, while letting the user know what's going on. This is where Gearman comes in handy. I've used message queues in the past, but they don't have the notion of progress like a job queue does. For example purposes, I've written a little script that fetches URLs and titles from a list of RSS feeds:

When memcache tells my application it can't find the droids it's looking for, my application now adds a background job to the gearman queue.

The application then forwards the user to a job status page, complete with a progress bar courtesy of the fantastic Dojo Toolkit. The user sits and watches in eager anticipation.

A gearman worker script is running on the server thanks to supervisord, it comes along picks the job from the queue and sets about fetching the data. As it progresses, it pings the gearman server about it's current status.

The page the user is glued to is repeatedly pinging the server to check on the progress of the job. As the job server gets updates from the worker, the web server passes them on to the user via the dijit.ProgressBar.

Once complete, the javascript forwards the user back to the original URL, the data is successfully fetched for the cache, and we're in business.

Get to the code already

You can browse the sample scripts at github, they're not amazing, just meant to show the method I've described here. As a side note, I wondered if anyone had come up with a Sinatra style framework for PHP, turns out there were quite a few, I chose the Slim Framework as I could see from the homepage it took PHP 5.3 lambdas, the others didn't make it too clear.

Has anyone else done anything similar? Am I doing something stupid? Comments are welcome.

Iterations plugin for Redmine

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Mon, 07 Feb 2011 14:11:34 nil

I was impressed by the simplicity of 37signals' Iterations when I saw their original post, it's a basic web app used to track their own internal ideas and feature requests, and also to help decide who's going to work on those features, but that was as far as I got with it, until I was reminded of it a few weeks later when listening to an episode of techzing¹.

[caption id="attachment_539" align="aligncenter" width="529" caption="More on the jabber bot later"][/caption]

I face somewhat of an uphill battle trying to encourage my staff to come up with their own ideas, initiatives and suggestions and thought something like this might help promote that kind of thing, so I set about creating a plugin for redmine, the issue tracker we use. I'm sure it's not quite as sexy as 37signals', but I'm quite happy with it and it's working out ok so far. [caption id="attachment_541" align="aligncenter" width="538" caption="Tracking ideas that aren't necessarily project based"][/caption]

I'm far from a skilled Rails developer, nevermind a redmine plugin developer, but if anyone would like to give the plugin a try, it's up on github. It's been developed against the redmine trunk, so your mileage may vary, but I would appreciate anyone giving the code a once over, tidying bits up, making it more rails/redmine friendly etc, and sending me a pull request.

[caption id="attachment_542" align="aligncenter" width="538" caption="Users can comment, +1 or offer to work on an iteration"][/caption]

¹ I recently started listening to podcasts again and I was very disappointed to find out StackOverflow have given up with the podcasting and You Look Nice Today are very few and far between, but after reading an inspiring blog post, I thought I'd give Jason's podcast a try. Slowly working my way through a few months worth of back catalog, quite enjoying it, worth a listen if you're into that kind of thing.

How to install gitolite on Ubuntu 10.10 (Maverick Meerkat)

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Sun, 05 Dec 2010 11:19:15 nil

At work we recently switched from subversion to Git for our version control. I wont go into it too much, but the main reasons where:

We wanted a distributed system, for the flexibility it offers individuals
We wanted the enhanced branch/merge
Just for fun really, broaden our horizons

Anyway, I love GitHub, but it's not the answer to everything! I wanted a central repository that I could control, so having had a brief glimpse at Gitorious and gitosis, I settled on gitolite. Now, I'm usually quite a lazy sys admin, and unless I desperately need a feature in the latest version of an application, I'm usually happy to fall back on the my chosen package manager, in this case Ubuntu's APT. Gitolite got a package as of version 10.10 Maverick Meerkat, so I told our local Ubuntu mirror to download all the 10.10 packages and after that, upgraded the server I had in mind so that I could use the gitolite package.

Install Gitolite

Nice and easy this part, on the server:

server> sudo apt-get update
server> sudo apt-get install gitolite

Creating a Public/Private key pair

If you already have one, send the public halve over to the server and skip this part. Otherwise, take a look at ssh key based authentication, then create your key pair on your client machine:

client> ssh-keygen 
Generating public/private rsa key pair.
Enter file in which to save the key (/home/davem/.ssh/id_rsa): 
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /home/davem/.ssh/id_rsa.
Your public key has been saved in /home/davem/.ssh/id_rsa.pub.
The key fingerprint is:
61:bf:f5:2d:f6:ed:cd:10:b7:0c:be:5d:4d:8f:a3:0d davem@client
The key's randomart image is:
+--[ RSA 2048]----+
|                 |
|                 |
|        o        |
|       . o       |
|        S . ... o|
|           o..o*+|
|          . E.Bo=|
|             =o*+|
|            ...o*|
+-----------------+

Once created, send the public halve to the gitolite server. Be sure to use the name you provided when creating the key. In this example, I've called the key davem.pub on the target machine, so I can differentiate between myself and other developers.

client> scp ~/.ssh/id_rsa.pub server:davem.pub

Configure gitolite

On the server, copy the public halve to a convenient location and run the gl-setup tool.

server> mv davem.pub /tmp/davem.pub
server> chmod 666 /tmp/davem.pub
server> sudo su gitolite
server> gl-setup /tmp/davem.pub
...

That's gitolite setup, we now need to go back to the client machine to fully configure it. First edit your .ssh/config file, so that ssh knows how to connect to the server. Again, be careful to use the correct name for your key pair:

Host servername
IdentityFile ~/.ssh/id_rsa

Now we can clone the git repository that is used to configure gitolite:

client> git clone gitolite@server:gitolite-admin
Initialized empty Git repository in /home/davem/gitolite-admin/.git/
remote: Counting objects: 6, done.
remote: Compressing objects: 100% (4/4), done.
remote: Total 6 (delta 0), reused 0 (delta 0)
Receiving objects: 100% (6/6), done.
client> cd gitolite-admin

Adding repositories

The gitolite admin contains two folders. The first, conf contains a single config file. Open that and create a new repository by adding:

        repo    mytest
                  RW+     =   @all

You then need to commit the changes and push them to the gitolite server:

client> git commit -m "Added mytest repo" conf/gitolite.conf
client> git push

We then should be able to clone our new repository:

client> git clone gitolite@server:mytest

Adding users

To add a new user, simply add their public key halve to your clone of the gitolite-admin repo, add, commit and push.

client> cd gitolite-admin
client> cp ~/Downloads/another.pub keydir/
client> git add keydir/another.pub
client> git commit -m "Added another as a user" keydir/another.pub
client> git push

I wont go any further than that, you can configure fine grain access control and other things in the conf/gitolite.conf file, check out the documentation. Hope it's been helpful, comments (especially corrections) are appreciated.

Notes from porting ruby to php

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Mon, 29 Nov 2010 22:13:30 nil

I read a nice post on porting c# code to php a while back via hacker news and found it quite interesting. This week I've found myself in a similar situation, only reversed, I'm porting from a language I'm not familiar with, and I'm porting code that I don't necessarily understand! I've written a couple of rails apps, but they were very basic, and I have no real clue with the delicacies of the ruby language.

We've recently switched to git and in order to help facilitate the flexibility that git brings, I thought I'd replace the database migrations system we currently use, predominantly to allow for 'interleaved' migrations, but the flexibility for running code as well as sql would be beneficial.

We evaluated some of the existing solutions out there and unsuprisingly we settled on rails' migrations feature set as being the best match for us. Now, I read about standalone migrations, but didn't really want to force our project to be dependant on another language, so decided to port ActiveRecord::Migration to php.

Update 02/11/2011: I've since moved on from this method of running migrations, checkout phpmig on github. It's similar to the tool I describe here, but not tied to a particular project, library or framework.

First I had to fathom out the file structure, everything seemed to be jammed into one file, so I converted that to a file structure that more closely resembled that of our application library. I also decided at this point that I wasn't going to bother with the automated reversal feature, that seems a little overkill for us. I also aren't really interested in abstracting the DDL at this point, SQL is all we need.

Spencer/
|-- Db
|   |-- Exception.php
|   |-- Migration
|   |   |-- Exception
|   |   |   |-- DuplicateName.php
|   |   |   |-- DuplicateVersion.php
|   |   |   |-- IllegalName.php
|   |   |   |-- Irreversible.php
|   |   |   `-- UnknownVersion.php
|   |   |-- Exception.php
|   |   `-- Migrator.php
|   |-- Migration.php
|   `-- MigrationProxy.php

Next I wrote the exception classes, converting something like this:

class DuplicateMigrationVersionError < ActiveRecordError#:nodoc:
  def initialize(version)
    super("Multiple migrations have the version number #{version}")
  end
end

to something like this:

class Spencer_Db_Migration_Exception_DuplicateVersion extends Spencer_Db_Migration_Exception
{
    public function __construct($version)
    {
        parent::__construct("Multiple migrations have the version number $version");
    }
}

That was simple enough, so then I moved on to the Migration class. This is where I hit my first roadblock and had to go learn some things about ruby meta-programming, specifically the singleton class. Now, I'll admit I still don't fully understand it, but worked out enough to see that the main use of the singleton class was to declare class methods. However, I was quickly stumped again by this piece of code:

class < < self
  attr_accessor :delegate # :nodoc:
end

def self.method_missing(name, *args, &block) # :nodoc:
  (delegate || superclass.delegate).send(name, *args, &block)
end

>>>> snip <<<<

# instantiate the delegate object after initialize is defined
self.verbose = true
self.delegate = new

def up
  self.class.delegate = self
  return unless self.class.respond_to?(:up)
  self.class.up
end

def down
  self.class.delegate = self
  return unless self.class.respond_to?(:down)
  self.class.down
end

Here's my very uneducated take on it.

We declare delegate as a class member.
We then override ruby's built in method_missing method, which will handle any messages sent to this class that it doesn't understand.
Set the verbose class member to true and the delegate member to a new instance of ActiveRecord::Migration, although I'm not too sure on the latter
Define two method instance methods, that set the delegate class member to the current instance, then call the same method on the class, which will delegate it to the instance we set as the delegate? - This just doesn't make sense to me, but maybe it's not supposed to? These are the two methods that the migrations themselves override

So anyway, I simply skipped that bit. I figured it would dawn on me what it was all for when I came to need it!

The next point of interest was the little Benchmark::measure method, which uses ruby's yield statement to measure the time taken to run a method. The method itself is passed as an argument to the method:

time = Benchmark.measure { send(direction) }

The Benchmark.measure method then records some times, runs the passed method by calling the yield statement, then records the finish times and returns them.

# File benchmark.rb, line 291
def measure(label = "") # :yield:
  t0, r0 = Benchmark.times, Time.now
  yield
  t1, r1 = Benchmark.times, Time.now
  Benchmark::Tms.new(t1.utime  - t0.utime, 
                     t1.stime  - t0.stime, 
                     t1.cutime - t0.cutime, 
                     t1.cstime - t0.cstime, 
                     r1.to_f - r0.to_f,
                     label)
end

Now I couldn't replicate this exactly using PHP, but the new closures available in PHP 5.3 made it relatively neat. I don't know of any way to measure cpu/user/real time in PHP, so I ended up simply with:

public static function measure(Closure $function)
{
    $time = microtime(1);
    $function();
    return new Spencer_Benchmark_Tms(microtime(1) - time());
}

The code to call it needs a little more explaining, the use keyword specifies variables from the parent scope that the closure can access.

$th = $this;
$time = Spencer_Benchmark::measure(function() use ($th, $direction) { $th->$direction(); });

Mimicking the Activerecord::MigrationProxy class was easy enough, then I moved on to the Activerecord::Migrator class. Now, the first problem I had was the use of method names in this class. Because of the aforementioned singleton class, ruby classes can effectively have both a class method and an instance method with the same name, which PHP can't. To keep track, I prefixed the instance methods with 'really' (not the best choice, but once I had started I thought it best to stick with it). Although it worked, it proved very difficult to keep track of as I was porting the code over. The next thing I noticed about ruby was how nice it was to be working with objects all the time. Using methods like Enumerable.map and Enumerable.detect just seems a little nicer than my own versions mangled up in PHP. Take the following code, that fetches a set of records from the database, uses map to convert the array of objects returned into an array of integers and then sorts it.

def get_all_versions
  table = Arel::Table.new(schema_migrations_table_name)
  Base.connection.select_values(table.project(table['version']).to_sql).map{ |v| v.to_i }.sort
end

My PHP equivalent gets the job done, but simply doesn't look as elegant, despite me cheating and using the SQL to sort the data. The SPL Iterators can run sorts with a user defined function, but I can't see map and detect etc.

public static function getAllVersions()
{
    try {
        $table = self::getDbTable();
        $rows = $table->fetchAll($table->select()->from($table, array('version'))->order('version ASC'))->toArray();
        return array_map(function($r) { return $r['version'];}, $rows);
    } catch (Exception $e) {
        return array();
    }
}

This seemed like a recurring theme as I ported the rest of Migrator, with the use of closures/blocks at every corner. Other than that, I didn't really have any problems. I skipped the DDL transactions, as, as far as I know, MySQL doesn't support them.

I'd then finished the library, so I took a look inside the rails rake tasks that are used to run the migrations. Thankfully, they were nice and simple and mostly called the nice class methods I'd already ported over to PHP. I've previously used phing for tasks like this, but I thought I'd give Zend_Tool a go. First, I'm not so keen on the way Zend_Tool integrates into a project. It needs a big fat XML file to work nicely, but that xml file can't be generated from existing code, so doesn't play well with a team of developers. Secondly, the zf command line program doesn't really tie in to your codebase at all. Ideally for me, the migration classes shouldn't need to worry about bootstrapping the application, so I copied the zf.php file to our codebase and customised it to bootstrap our application and also set the include path up for the custom provider I was about to write. An hour later and I had a nice Zend_Tool provider, that nicely generate, ran and provided status on migrations.

davem@CSL2717:[develop]sos$ ./zf ? migration
Zend Framework Command Line Console Tool v1.10.7
Actions supported by provider "Migration"
  Migration
    zf status migration
    zf migrate migration version
    zf redo migration verison
    zf up migration version
    zf down migration version
    zf forward migration steps[=1]
    zf rollback migration steps[=1]
    zf generate migration name


davem@CSL2717:[develop]sos$ ./zf generate migration FirstMigration
Created migration at 20101129205655_first_migration.php

davem@CSL2717:[develop]sos$ vim db/migrations/20101129205655_first_migration.php 
davem@CSL2717:[develop]sos$ ./zf status migration

 Status   Migration ID    Migration Name
--------------------------------------------------
   down  20101129205655  FirstMigration


davem@CSL2717:[develop]sos$ ./zf migrate migration
== 20101129205655 FirstMigration: migrating ===================================
== 20101129205655 FirstMigration: migrated 0.3273s ============================

davem@CSL2717:[develop]sos$ ./zf status migration

 Status   Migration ID    Migration Name
--------------------------------------------------
     up  20101129205655  FirstMigration


davem@CSL2717:[develop]sos$

The code isn't really in any state to be published, needs a lot of polishing, but once that's done, I'll have a word with my bosses and maybe sling it up on github.

In conclusion, I can see while all the ruby fanboys become fanboys, but while it doesn't always look as pretty, php still gets things done for me.

ZFSnippets.com now managed by bescript.de

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Tue, 01 Jun 2010 13:18:41 nil

Not so long ago, I posted requesting that people who fancied maintaining ZFSnippets.com should get in touch and see if I could hand it over. Thanks to everyone who volunteered (sorry I didn't get back to you all individually!) and eventually I decided to hand it over to German PHP outfit, bescript.de. Ben was very clear and excited in his email and after discussing it with him I knew the handover would be nice and easy and bescript.de would take good care of the site. They have already added new features and ticked some items off the uservoice list!

Again, thanks to everyone who got in touch and best of luck to Ben and his team with the site.

Set MySQL connection variables with mysql-proxy

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Tue, 01 Jun 2010 11:01:02 nil

We recently moved the application I work on everyday to Amazon Web Services and bravely adopted their Relational Database Service (RDS) and have had little trouble thus far, but the other day I noticed since we kicked into BST, timestamps in the database where an hour behind. Low and behold, the default time zone cannot be changed. Luckily, we've been using mysql-proxy since we migrated and rather than changed our application, I managed to knock up a lua script that sets the timezone variable on every query. It would be nice if it could do it when it creates a connection, but I've not worked out how to do that yet!

---
-- read_query() can rewrite packets
--
function read_query( packet )
        if string.byte(packet) == proxy.COM_QUERY then
                proxy.queries:append(1, string.char(proxy.COM_QUERY) .. "SET time_zone = 'Europe/London'", {resultset_is_needed = true})
                proxy.queries:append(2, packet)
                return proxy.PROXY_SEND_QUERY
        end
end

---
-- read_query_result() is called when we receive a query result
-- from the server
--
function read_query_result(inj)
        if (inj.type == 1) then
            return proxy.PROXY_IGNORE_RESULT
        end
end

If anyone could point me in the general direction for setting the variable at connection time, it'd be appreciated. I assume I can create a create_connection function, but I don't know where to go from there.

Wanted: New home for zfsnippets.com

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Mon, 26 Apr 2010 10:58:22 nil

Update 27/04/2010: I have received a number of enquiries via email and I'll be going through them to try and find the best candidate, I'll also make an effort to reply to all emails I've received. Thanks to all for your interest.

It's been just over a year since I initially launched zfsnippets.com, it was a good little project for me to get used to the Zend Framework, but since then I've totally neglected it. I'd normally leave a website going despite my lack of enthusiasm, but I no longer need the VPS it is hosted on so I'm looking for someone else to take the project on and move to their hosting solution.

My interests have moved quite rapidly in the last year and the project is sadly no longer of interest to me. ZFSnippets receives approximately 2,500 visits a month, so I think it's worth keeping the project alive.

If you're interested in hosting the site and hopefully building on it and improving it, please email me, dave.marshall at atstsolutions.co.uk.

Review: Zend Framework 1.8 Web Application Development

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Thu, 11 Feb 2010 14:51:24 nil

Sometime last year, I, along with quite a few others, was asked to review one of Packt Publishing's new books, Zend Framework 1.8 Web Application Development, written by Keith Pope. They sent me a copy, which was very good of them and although it's taken me ages to finish and get round to writing this review, that's not a true reflection of how good the book was, I'm just a very busy/lazy person! So lazy, that I did in fact say I'd have it done in two weeks, which turned into 4 months.

Packt asked if I'd be interested in reviewing the book, so watch this space, I'll be back in a couple of weeks with a review.

My Post dated 15/10/2009

Introduction

Design, develop, and deploy feature-rich PHP web applications with this MVC framework

That is the books strap line, and it does exactly what it says on the tin. The bulk of the book actually takes you through the design, development, testing and deployment of a real world example application, called the Storefront. The book claims that it is written for PHP web developers that are either using or looking to start using the Zend Framework and that a basic knowledge of Object Oriented design would be helpful. While you might be able to manage without any OOD experience, I'd say you definitely need some to get the most out of this book, as the second chapter digs right under the hood of the Frameworks MVC architecture. My personal experience was that I got to learn all the things I haven't had time to learn, I've been using the Framework for a couple of years now, always appreciating, but not always understanding what it was doing for me.

MVC Architecture

The first chapter gives you a brief overview of creating an MVC application in the Zend Framework, experienced users of the Framework will probably want to gloss over this part, whereas people looking to start using the framework should take their time and take things in. The next chapter is when I really started to enjoy the book. Each component of the MVC architecture is presented as it's own topic, with each component getting a breakdown of Design Patterns/theory, default settings/configuration, usage and finally customisation.

The chapter is well put together and considering the amount of information portrayed, is not overwhelming.

Storefront Application

The rest of the book provides the information you need about the framework around a real world example application, called StoreFront, which is a basic e-commerce application. I should point out that I didn't code the application as I went, if I'm reading I like to read, but where appropriate I have used the book as a reference when updating my existing Zend Framework applications.

The best thing about these chapters though, is some of the design theory you pick up on the way, that isn't directly relevant to the Zend Framework, but can be applied to any framework out there. Best practices such as Fat Models, Composition, Fluent interfaces are all explained in detail, along with relevant and realistic examples. Further more, the applications MVC separation is excellent, taken in context (it might be a little overkill for the example application, but is there to show you the methods).

After taking you through the creation of the application, the book then takes you into optimisation and testing. The optimisation takes you though some general PHP optimisation techniques, but then ploughs into techniques like a transparent abstract cache that is applied to the models. Testing is carried out with the trusty PHPUnit, along with the frameworks extension of the library Zend_Test and the book goes on to integrate the test suites with apache ant (why not phing) and phpundercontrol.

Conclusion

In conclusion, I thought this book was an excellent read and I plan to follow it through again when I build my next ZF app (I have two good ideas in the pipeline). Find out more or and buy it!. Thanks to Packt for sending me a copy!

Zend Framework 1.8 Web Application Development

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Thu, 15 Oct 2009 08:55:04 nil

Packt Publishing have recently contacted me letting me know about one of their new books, Zend Framework 1.8 Web Application Development . It looks reasonably priced, and if you fancy having a quick look before you by, the author Keith Pope has a free chapter to download. Packt asked if I'd be interested in reviewing the book, so watch this space, I'll be back in a couple of weeks with a review.

Zend Framework Poster

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Fri, 21 Aug 2009 11:18:59 nil

After stumbling across an offer for a free Zend Framework Poster some time ago, I quickly dropped Mayflower an email.

Some time passed, and now it's here at work, placed next to our all important tea, coffee and biscuits station.

It's A0, covers nearly all of the major components and is well worth having! Top props to guys over there for pushing the Zend Framework like this. Cheers!

Mozilla's Content Security Policy (CSP)

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Tue, 30 Jun 2009 09:53:04 nil

I saw this post via SlashDot and can't help but think it's a little overkill?

Content Security Policy is intended to mitigate a large class of Web Application Vulnerabilities: Cross Site Scripting. Cross Site Request Forgery has also become a large scale problem in Web Application Security, though it is not a primary focus of Content Security Policy.

In an ideal world, this would be great, but getting all the browsers on board and implemented may take a while. I was thinking about this the other day and I don't see why the browsers/w3c can't standardise on some sort of tag or conditional comments that says don't execute any script in here. This would be simple to use and surely simple to implement. Browsers already know what to do with <noscript>

For Example:

<dontexecutescript>
    <?php echo $this->escape($userProvidedContent);?>
</dontexecutescript>

Or:

<!--[dontexecutescript] -->
    <?php echo $this->escape($userProvidedContent);?>
<!--[dontexecutescript]-->

I'm no expert on XSS, but I'm pretty sure this would solve most of the issues encountered.

Update:

Okay, so one obvious problem might be that the $userProvidedContent contains a closing </dontexecutescript> tag, but that's just semantics. Unique identifiers for each block, ignoring tags that don't match up, these browser developers are clever, they could come up with something.

Using message queues to improve user experience

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Mon, 01 Jun 2009 13:22:48 nil

A major part of the application I develop on my day job is a big DMS, part of which is the ability to distribute documents to staff and external parties. The distribution system works in such a way that, if sending a document to 300 people, there will actually be 300 individual emails created, rather than one email with a list of recipients. This is desired behavior. My problem is, sending 300 emails at the click of a button can take a little time, degrading the experience for the users. This portion of the call graph shows sending to just five people was taking 1+ seconds.

To solve this, I started out to implement a simple queuing system, whereby the distribution requests are added to a queue, before being sent out by a scheduled task.

Rather than refactor a lot of code and do this some fancy way, I quickly put in a solution that proved the concept and seems to work pretty well for now, with minimal effort. As an example (our code's slightly more complex, I don't get paid for nothing), here's what I started with:

< ?php
class EmailSender {
    /**
     * Takes an array of addresses and sends an email to each one.
     *
     * @param array $address
     */
    public static function sendEmails($address) {
        /**
         * Code in here
         */
    }
}

The idea was fairly simple and I'm sure it's been done many times before. First step was to rename the existing method and make it private. I then wrote a new method, that checked to see if there was a queue available, if so adds the request to the queue, otherwise calls the old method. Then all I had to do was write a method that checks the queue, running any requests it finds through the original method. We've recently adopted the Zend Framework, so checking out Zend_Queue from the incubator, reading some documentation with my docbook goggles on (couldn't be bothered to build it) and it was pretty much in place.

<?php
/**
 * Zend_Queue offline processing hack example
 *
 * @author      Dave Marshall 
 * @version     $Rev: $
 * @since       $Date: $
 * @link        $URL: $
 */
class EmailSender {

    private static $queue = null;

    /**
     * Set Queue
     *
     * @param Zend_Queue $queue
     */
    public static function setQueue($queue)
    {
        self::$queue = $queue;
    }

    /**
     * Takes an array of addresses and sends an email to each one.
     *
     * @see reallySendEmail
     * @see sendQueuedEmails
     * @param array $address
     */
    public static function sendEmail($address)
    {
        if (self::$queue === null) {
            return self::reallySendEmail($address);
        }

        self::$queue->send(serialize(func_get_args()));
    }

    /**
     * Takes an array of addresses and sends an email to each one.
     *
     * @see sendEmail
     * @param array $address
     */
    private static function reallySendEmail($address) 
    {
        /**
         * Code in here
         */
        echo 'Sending email to ' . implode(', ', $address) . PHP_EOL;
    }

    /**
     * Reads emails from the queue and sends them
     *
     * @param int $count - The number of queued items to process
     */
    public static function sendQueuedEmails($count)
    {
        /**
         * Should really check the queue is good here
         */

        $messages = self::$queue->receive(intval($count));
        foreach($messages as $msg) {
            $args = unserialize($msg->body);
            call_user_func_array(array(__CLASS__, 'reallySendEmail'), $args);
            self::$queue->deleteMessage($msg);
        }
    }
}


set_include_path(
    dirname(__FILE__) . '/src/Zend_Framework/library' . PATH_SEPARATOR
    . dirname(__FILE__) . '/src/ZendI/library' . PATH_SEPARATOR
    . get_include_path()
);

require_once "Zend/Loader.php";
Zend_Loader::registerAutoload();

define('DB_SERVER', 'localhost');
define('DB_PORT', 3306);
define('DB_USER', 'root');
define('DB_PASS', 'password');
define('DB_NAME', 'queue_example');

/**
 * Transmittal Queue
 *                   
 */
$config = array(
    'name' => 'transmittal',
    'driverOptions' => array(
        'host'     => DB_SERVER,
        'port'     => DB_PORT,
        'username' => DB_USER,
        'password' => DB_PASS,
        'dbname'   => DB_NAME,
        'type'     => 'pdo_mysql'
    )
);

// Create a database queue
$queue = new Zend_Queue('Db', $config);
$queue->createQueue('myqueue'); // called for good measure

EmailSender::setQueue($queue);

/**
 * Usage for adding to the queue
 */
EmailSender::sendEmail(array('davemastergeneral@gmail.com'));

/**
 * Usage for scheduled task
 */
EmailSender::sendQueuedEmails(5);

This may not be the best practice in the world, but it got the job done Check it out at ZFSnippets.com

Redmine Issue Tracker/Project Management

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Mon, 20 Apr 2009 15:30:42 nil

We recently started using Redmine as our issue tracking software and we're very pleased with it. I trialled several alternatives, both open source and commercial, including Jira, FogBugz, Mantis and Trac, but decided on Redmine. Jira and FogBugz look like excellent products, but I don't feel we would use the extra functionality that warrants the price tag. Mantis looks a bit dated and although it's been a while, last time I tried to install and configure Trac, it was a bit of a nightmare.

First thing first, here are some of our requirements for an issue tracker:

Multiple Projects
Time tracking
Notifications (email, rss, im etc.)
Wiki
SCM integration

Redmine does all of this and a little more, but we haven't used the other features that much. It does what we want really well, so that's good enough for us.

What I like

Editing issues in bulk is fantastic. We deploy our application fairly casually, and when we do we set all the issues that have been deployed as closed and assign them a version number. Highlight the issues, right click and set the version number.

The SCM integration does all we need it to. If we add Fixes #500 or Implements #500 to a commit message, issue 500 will be set to resolved and 100% complete and the revision number and commit message appears on the issue's page. Alternatively, just putting Refs #500 will show the revision number and commit message in the page, without changing the status.

Issues themselves can have custom fields, but we don't use them all that much. We do make lots of use of the descriptions and notes as they both take wiki markup, meaning we can enter rich text including source code highlighting and images, linking to changesets, wiki pages, target versions, source code and other issues. I like the ability to embed screenshots and call graphs in issues.

What I don't like

Here's a couple of things that bother me, if I shift my arse I might look into creating patches for the simple ones.

No API! It would be really nice to have an API and there's one in the works, but it's sounds like it's a bit of a ballache changing fat controllers into fat models.

I often wish a few more items where 'clickable'. For instance when viewing the issue list, I'd like to be able to click the Category name in one of the rows and see a list of issues for that category.

You can save filters on the issue list, which is ace, but it would be nice if you can 'save as new'. I often create lots of very similar queries and have to build up each one from scratch.

Rev=Canonical and all that Jazz

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Wed, 15 Apr 2009 11:58:29 nil

If anybody missed it, the last few days has seen plenty of buzz around a new proposal on how to solve the problem with URL shorteners. I kind of got lost in all the different methods and proposals people are discussing, suggesting or implementing, but I went ahead and added some simple logic to lnkd.in, to do a HTTP HEAD request to the given URL, looking for headers in a couple of the formats suggested. I figured that was going to get out of date pretty quickly, so I modified it to use the RevCanonical API, seems to work pretty well, returning a rev=canonical url wherever possible.

I also contributed a basic bit of code to Rob Allen's Shorter Links plugin for wordpress, allowing users to specifying a base url, davedevelopment.co.uk isn't all that good for short URLs. Just need to upgrade the plugin and decide on a short domain for my blog now.

Update: registerd daved.in, works a treat

Url Shortener in CodeIgniter

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Fri, 10 Apr 2009 00:38:52 nil

After seeing a response to a thread on Hacker News, I thought I'd have a crack at making a simple little URL shortener with CodeIgniter. I've never used the framework before and it was a nice quick little app to implement and get a feel for things. It's not quite worthy of competing with Bit.ly or Tinyurl.com, there's no checking for spam urls etc, but if you want to take a look, the source code is available, as is a live demo at lnkd.in.

ZFSnippets.com update

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Mon, 06 Apr 2009 21:17:13 nil

It's been just over a month since I launched zfsnippets.com and I'm very pleased with the reception it got, which spurred me on to try and improve it.

Since then, following suggestions on the feedback forum I've implemented a few new features. What I've noticed is the ease at which these features where added, thanks to the quality of the Zend Framework.

Favourites

Along with adding icons for up votes, there's now a little star icon to add snippets to your favourites, easily accessible from your user page. A little sprinkling of dojo and these are added via AJAX.

Revision History

Snippets can now be edited by the creator and a revision history is kept.

Using a simple implementation of the Longest common subsequence problem, you can also view diffs of the revisions.

Search

Lastly there's the search engine, built on top of Zend_Search_Lucene. This took a little more work, but the results were worth it. I hadn't realised how powerful the Lucene query language was until now and I'm very impressed with the implementation.

As you would expect, traffic has slowed down considerably since the initial burst, but there's still a bit of steady traffic there and I hope people keep coming back to visit now and then. As for going forward, I think I'll add a field to allow contributors to specify which zend framework version their snippet was tested or is compatible with, then look a little more at the scoring system.

Code Complete by Steve McConnell

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Wed, 25 Mar 2009 11:00:06 nil

The only programming book I use without fail every working day ;)

Seriously though, this book is great.

ZFSnippets.com - Zend Framework Code Snippets

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Wed, 04 Mar 2009 13:25:04 nil

Not so long ago I was working on a little project using the django framework and was looking for a simple nested set implementation, I found one that was good enough for my needs on djangosnippets.org. I thought this was pretty cool, and realised we didn't have a one stop shop for Zend Framework code snippets. Symfony and CakePHP also have dedicated sites for this. I'm aware of generic snippets sites and pastebin sites, but I thought it'd be another good learning opportunity to build my own with the Zend Framework, for the Zend Framework.

I've ended up implementing a pretty simple site, ZFSnippets. The current basic implementation makes use of:

There are plenty of things I'd like to implement (and fix;)), but will have to see if I find anytime:

A site search engine using Zend_Search_Lucene
More flexible posting, probably using wiki style markup
Version history for all snippets
Moving to a stackoverflow.com style, trust model, allowing regular users to collaborate, edit and update snippets.
Forgot my password link
Contextual error handling and descriptions
Accurate scoring/ranking, based on complex calculations.

I did actually spend quite a bit of time looking at markup libraries, coupled with HTMLPurifier, but it seemed like too much effort to start with, so I opted for a method of splitting the source code using hashes like pastie.

I like the idea of having a go with Zend_Search_Lucene, probably adding particular snippets to a queue for indexing whenever somebody posts one or comments on one.

Having built the site, I'm actually struggling to think of what might go on there. Helpers seem like an obvious one, but without building large, complex Zend Framework application, I don't have much need for custom code, the framework does the leg work for me. Nevermind, if you think it might be any use to you, head on over there and have a browse around, although it's a little low on content at the minute, or maybe subscribe to the feed and keep your eye out for anything interesting.

How protected is protected?

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Wed, 25 Feb 2009 22:18:21 nil

Before my much needed Holiday, a colleague of mine asked for my input on a funny issue he was having. The root of the problem was that our production server is still running PHP 5.1.2, where as most of us run 5.2.x on our development and test machines. My problem was that what I was seeing didn't make sense. The following portion of code is a simple reproduction of the code my colleague was using, running fine on PHP 5.2.x, but causing a fatal error on the production server running 5.1.2, an attribute access violation.

<?php
class BankAccount
{
    protected $balance;

    public function __construct($balance)
    {
        $this->balance = $balance;
    }

    public function __toString()
    {
        return 'Balance: ' . $this->balance;
    }

    public function debit($debit)
    {
        $this->balance -= $credit;
    }

    public function credit($credit)
    {
        $this->balance += $credit;
    }
}

class SavingsAccount extends BankAccount
{
    public function __construct(BankAccount $parent)
    {
        $this->balance = $parent->balance;
    }
}

$first = new BankAccount(1.00);
$second = new SavingsAccount($first);

echo $second, PHP_EOL; // Balance: 1

It turns out that PHP's visibility restrictions are on a class level, rather than an instance level, and the fatal error was actually due to a bug in 5.1.x.

The visibility of a property or method can be defined by prefixing the declaration with the keywords: public, protected or private. Public declared items can be accessed everywhere. Protected limits access to inherited and parent classes (and to the class that defines the item). Private limits visibility only to the class that defines the item.

This bemused me, as it would appear to me there is no point having a protected operator at all. Suppose I am a lowly programmer given the BankAccount Class above as an API I can use but not change. I have a data access object that returns BankAccount objects too, which I also can't touch. For reasons unknown to me, but probably to try and create some encapsulation so they can log transactions via the credit and debit methods, the original developer decided that I shouldn't be able to directly access the balance attribute, but I want to.

<?php

//$myAccount = $dao->getAccount(123);
$myAccount = new BankAccount(1.00);
echo $myAccount, PHP_EOL; // Balance: 1

class WorkAround extends BankAccount
{
    public static function setBalance(BankAccount $acc, $balance)
    {
        $acc->balance = $balance;
    }
}

WorkAround::setBalance($myAccount, 1000000.00);
echo $myAccount, PHP_EOL; // Balance: 1000000

This just doesn't seem right to me. Don't get me wrong, I appreciate there needs to be some level of responsibility taken by developers to not do this kind of thing. My colleague writes a lot of Java, in which this is also the expected behaviour with the protected visibility, hence why he set out on this path, so it seemed right to him. Because of this I embarked on a little research to determine how other languages implement visibility.

A very quick and non-extensive bit of research led me to believe that Python doesn't have visibility as I know it, and C# has protected and internal, but neither work the way I'd like. Ruby has the closest to what I desire in the form of instance variables, but I'm sure there's plenty I've missed.

class BankAccount
    def initialize(bal)
        @balance = bal
    end

    def debit(debit)
        @balance -= debit
    end

    def credit(credit)
        @balance += credit
    end

    def to_s
        "Balance: %d" % @balance
    end
end

class SavingsAccount < BankAccount
    def setBalance(acc, bal)
        # This wont work - we cant access acc.balance
        # acc.balance = balance;
    end

    def bonusCredit(credit)
        # just to prove a sub class can access 
        # the instance variable
        @balance += credit + 1
    end
end

first = BankAccount.new(42);
second = SavingsAccount.new(1);

second.setBalance(first, 5000000);
second.bonusCredit(10);

print first  # Balance: 42
print second # Balance: 12

It worries me sometimes when I come across things like this that I blatantly should know and understand properly. Cheers JC.

Competition: PHP Job Hunters Handbook up for grabs

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Mon, 05 Jan 2009 18:03:25 nil

I've got a couple of copies of Php|architect's PHP Job Hunter's Handbook to give away, the only catch is you have to sign up to PHPPositions' feed via email. It's managed by Feedburner, so it can be trusted and you wont get any spam, just super smashing great php jobs in the UK.

I'll pick two email addresses at random this time next week and get the books posted out.

Meeting Cost Clock

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Wed, 24 Dec 2008 13:06:03 nil

I wrote a simple Meeting Cost Clock after having a little search on google for a simple one (I actually found this one after writing mine). It's easy to use, you enter the number of participants, the average salary for those participants, then click start. It's a little rudimentary, the JavaScript is probably a little rough, but it'll serve me right.

I've styled it using iPhone universal, a CSS framework for making iPhone stylee pages, so one of my colleagues can open it up on his iPhone in our next meeting.

Landing a PHP job Part 3: Curriculum Vitae

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Mon, 15 Dec 2008 23:44:24 nil

In part two of this series, I discussed the technical know how I think will help get you your next PHP job. This part will discuss writing your Curriculum Vitae(CV, resume, etc.). There are a lot of contrasting opinions on this subject, I'll make a few points, give you some further reading and you can adapt the opinions in to a top notch CV of your own. I'm no major expert and most of the recruitment I have been involved in has been for trainee developers, but these positions attract a high number of CVs, so I've seen a fair few.

Your CV does not get you a job

Your CV gets you an interview, your performance in the interview gets you a job. Your CV is a right of passage, this stage is used to filter out the wrong candidates.

Your CV should evolve like you

You should be continually evolving and improving yourself, your CV should continually evolve with you. I can't see any reason why any two companies should see the same version of your CV. Every time you apply for a position, you CV should be tailored to suit the position. Cut out anything you think will not interest your prospective employer, embellish on what will interest them. You come across as a better candidate and you don't waste their time.

Don't stuff your CV with keywords/acronyms

Skills: PHP4/5, SOAP, XML, XSLT, JSON, AJAX, (X)HTML, CSS, RoR, MySQL, SEO, WAI, WCAG, MVC, XML-RPC....

These kinds of lists are great for getting your CV past an agency recruiter, but the actual employers would rather see a reasonable description of how you used 5 of those technologies. I try to briefly describe what I did and why I used those methods/skills/technologies.

.. Overcame performance issues due to large volumes of data by including caching, AJAX and moving some business logic to database triggers and stored procedures. (LAMP)

Besides, if you're good, they'll hire you and expect you to quickly learn the skills, technologies and methods they use.

Formatting and Proof Reading

I like CVs short, they take less time to read. One page is good, any more than two is bad. Keep it simple, spell check it, grammar check it, get people smarter than you to proof read it. Speaking of which, here's my current offering, although it still needs a lot of work. I intend to try switching to plain text, ala Stevey, plus I recently got promoted so I've more work history to add. Comments appreciated.

Changing hosts - Slicehost.com Review

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Tue, 11 Nov 2008 15:20:02 nil

I'm currently trying to sell DaveProxy and as such will be switching to a more cost effective hosting solution.

I've been using dedicated servers for quite some time now, initially with LayeredTech, but after their atrocious price increase, I promptly moved to iWeb, who I would recommend.

Moving away from the dedicated server arena I start looking at Virtual Private Servers, which give all the benefits of running your own dedicated server, usually at a fraction of the price, along with a few added benefits. This lead me to Slicehost.com.

Slicehost

Slicehost were a name I'd heard of, so I went to check them out. Their tagline, 'Built for Developers', just about sums it all up for me. With plans starting from just $20/month, I was instantly interested and their well designed website answered all the questions I needed. The features that did it for me the most:

Ajax console access - The management portal has an Ajax terminal, just in case your SSH goes down
No contracts, no setup fees. - Always a bonus
Upgrade, downgrade, add a slice or remove a slice anytime. - This is a thing of beauty. Because your VPS is essential a disk image, if you decide you need more RAM, disk space or bandwidth, you can simply upgrade via the management portal. Your VPS will be taken down and fired back up in a more powerful virtual machine.
Backups - Again, because your VPS is stored as a disk image, starting from $5/month, you can have upto 3 snapshots (1 daily, 1 weekly and 1 variable) of your VPS available to restore at any time.

Another bonus, they were recently acquired by Rackspace, so you can rest assured they are fairly stable and not going anywhere.

My Slice

Ordering a slice was nice and easy and what really impressed me was the time it took for my slice to be ready to use. I'm used to a 48 hour wait on new dedicated servers, but as you can imagine, creating a new virtual machine takes a matter of minutes. After confirming my details I was taken to the management portal, which showed my slice as 60% built. Two refreshes later and it was complete, I checked my email and my user credentials were waiting for me.

The management portal is also excellent, it's really nice and basic in terms of looks, no fancy gradients.

This blog is the only site I have moved over so far, it was nice and simple as the slice is running Ubuntu like my dedicated server.

Only time will tell how reliable they are, but I'm extremely impressed so far.

Going to PHPNW08

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Mon, 03 Nov 2008 15:58:12 nil

In a few weeks a few colleagues and myself will be attending phpnw08, a PHP conference arranged by the PHPNW community.

phpnw08 is a 1 day conference, to be held on Saturday 22nd November 2008, for developers, designers, managers or anyone else with an interest in the PHP programming language.

The conference will have a range of well known as well as more local speakers and aims to highlight current best practice and emerging topics within the sphere of PHP and web development.

This will be the first conference I've attended and I'm lucky enough to have my ticket paid for by Spencers. I'm particularly looking forward to the keynote with Derick Rethans, The Power of Refactoring with Stefan Koopmanschap and the final panel with Steph Fox, Ivo Jansch, Scott Macvicar and Felix De Vliegher.

Anybody else who's going, that's my picture over on the right, if you see me, please introduce yourself! Anybody who's in the UK and not going, register now!

Programming in C and stackoverflow.com

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Thu, 30 Oct 2008 14:16:55 nil

I recently had to stroll from the PHP path, at work we required a very simple program that had to be run as setuid. Now, as far as I can tell, both debian and ubuntu do not allow scripts to setuid, only binaries. This led to me flexing my C skills, which haven't been touched since I was following NeHe's OpenGL tutorials for my graphics module back at uni. I couldn't actually remember how to do anything, so I had a quick read through a few chapters from my namesakes guide (I'm gunning for his place in the google rankings for my name) and set about on my task.

I managed to do everything I needed using the guide, except for one simple task. In PHP, we have the file_exists function, which works a treat. There's no such function in C, which is understandable as it is a much higher level language, so I had to hit the search engines to find an example. The result at the top of the list was actually a forum post asking the exact same question I was asking google. Now being the naive person I am, I took the first correct response as gospel and implemented it, it worked and I was happy. Now, a few hours later my program was finished and I as read through the code again, the file_exists function I'd created was nagging at me. The function basically ran fopen on the file, returning true upon success, which I assumed was slightly overkill.

I've been keeping my eye on stackoverflow since the first announcement, including listening to all the podcasts and participating in the private beta. Joel Spolsky and Jeff Atwood have done extremely well in building a giant community of programmers and I figured I'd get the answer I needed there. I asked my question and in moments I had several answers. Turned out that there were two methods better than my own and I had to make a decision and award the 'correct' answer to one of them. Even so, the other answer was well voted up, so what I considered the second best response, was directly under my accepted answer. A week later and I thought I'd check the google rankings, to see how close to the top my question would be. It could be closer, but a search for file exists in C shows my question just on the first page and I'm sure with time, will make it to the top. In doing so, I had another look at the original forum thread that I got the fopen code from and low and behold, the last three posts detailed the same two answers that I got from stackoverflow. The difference? When people go to my stackoverflow.com question, the best solution is the first thing they see after the question.

Zend Framework and the Twitter API

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Mon, 13 Oct 2008 14:16:58 nil

I wanted my new job website to post a tweet to twitter every time we approved a posting.

Zend_Service_Twitter looks like it will be fairly comprehensive, but it's not in the core yet and is probably a little overkill for my simple use case.

I then had a look at Zend_Rest_Client, which seemed to confuse me. I couldn't actually get it to add the parameters I wanted to the call, I guess it's better for interacting with Zend_Rest_Server or fully restful APIs.

To be fair, the manual actually states:

[Warning] Strictness of Zend_Rest_Client

Any REST service that is strict about the arguments it receives will likely fail using Zend_Rest_Client, because of the behavior described above. This is not a common practice and should not cause problems.

So here's some simple code using Zend_Http_Client.

< ?php
require_once 'Zend/Http/Client.php';

$http = new Zend_Http_Client('http://twitter.com/statuses/update.xml', array(
    'maxredirects' => 0,
    'timeout'      => 10,
));

$http->setAuth(
    'twitter_username',
    'twitter_password',
     Zend_Http_Client::AUTH_BASIC
);

$http->setMethod(Zend_Http_Client::POST);
$http->setParameterPost('status', 'Your status message');
$http->request();

?>

PHPPositions - Genuine PHP jobs at Genuine Companies

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Mon, 29 Sep 2008 21:51:03 nil

PHPPositions UK is a simple job board, listing genuine PHP jobs, for genuine companies. No agencies.

I needed a little project to help get to grips with the Zend Framework, so I created a little job board specifically targetted at PHP jobs in the UK. It's not finished yet, but it's good enough to put up on the web.

Check it out, if you're in the UK and on the lookout for positions, subscribe to the feed. Eventually I'd like to monetise the site, but that depends on it generating a large enough audience, so until that point I'll be adding jobs manually, thus keeping up a fairly high standard of jobs on there.

Quick SVN stats one liner

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Fri, 26 Sep 2008 11:00:29 nil

I needed a quick list of contributors to our repository, this one liner is nice and easy.

svn log -q | awk '/^r/ {print $3}' | sort | uniq -c

Landing a PHP job Part 2: Soft Skills

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Tue, 16 Sep 2008 23:23:20 nil

In part one of this series, I discussed the technical know how I think will help get you your next PHP job. This part points out some of the soft skills that are required for software development, discussing reasoning and how to go about acquiring those skills.

Writing

Like it or not, PHP developers are going to have to write documents of some kind as part of their job. You need to produce clear and concise documentation, including but not limited to requirements specifications, design specifications, technical documentation, end user documentation and possibly evening marketing documents and copy depending on the size of your company. Three quarters of this post is about communicating in different forms, getting your ideas across on paper is the first step to being better than the average programmer. Joel Spolsky actually recommends programmers take a creative writing course.

Listening

The guys at Manager Tools had an episode on teaching people interpersonal skills a while back and they made the point that being a good speaker is great, as long as you've got something great to say. If you can't listen to other people, you'll never have anything good to say. That cast is worth checking out, the points made are easily applied to yourself rather than other people.

Another nice article, this one concentrates on the barriers we all face while trying to listen to people, read it and be concious of them. The first time you put them into practice might be an interview, which you can't afford to mess up.

You're going to have to listen to lots of people, people you report to, people who report to you and clients. The better you are at listening, the better you'll be at your job.

Speaking

When it comes to speaking, I find the most important rule is to know what you want to say. Sometimes we're forced into situations where we have to act and speak on instinct, but these situations are the minority. Most of our speaking comes in the form of meetings and presentations, both of which are usually arranged in advanced. If they are arranged in advance, you should have time to plan what you want to say in advance and also what you want to get out of it. Even if things don't go quite according to plan, the fact that you had a plan should have at the very least raised your confidence. Confidence can be quite a key factor when speaking, many things affect our confidence. For example, in my eyes an easy win is to look smart. If you look smart, or at least feel like you look smart, your confidence will be higher and the way you communicate will be better.

Time Management

Productivity is key, whatever you do. I'm a big fan of Getting Things Done and I use Remember the Milk to manage my tasks. LifeHacker has a nice article on GTD with RTM.

While not strictly a soft skill, without being able to give reasonable estimates for development work, you wont be able to manage your time effectively. Software estimation is exceptionally difficult, this book comes highly recommended, but if you want some lighter reading, I recommend Joel Spolskys Painless Software Schedules. That post actual says it is obsolete, but I don't think it's true, the Evidence Based Scheduling article takes things to another level, but the base techniques are still the same.

Influence

I'm part way through this book and I'm yet to put it to much use, but it's a bloody good read. Being able to influence people comes in handy in all walks of life, selling software, recommending the latest technology to your boss or peers or convincing your boss you really need to go to ZendCon.

That's it for now, I'll be back with part 3 soon, which will be on writing your CV/Resume.

Landing A Php Job Part 1 Technical Knowledge And Skills

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Mon, 08 Sep 2008 00:00:00 nil

wordpress_id: 101 layout: post title: Landing a PHP job Part 1: Technical Knowledge and Skills date: 2008-09-08 22:28:28 +01:00 wordpress_url: http://davedevelopment.co.uk/?p=101

--- After reading this thread, I thought I'd spend some time writing about what I feel are some measures you can take to landing a job in PHP. This first part is going to concentrate on the kind of technical matters I think any PHP developer should at least have knowledge of, if not some kind of experience. A lot of the subjects discussed aren't specific to PHP, but the focus will be on PHP. It'll be far from exhaustive (please feel free to flame, but constructive comments would be nicer) and there'll probably be quite a few references to Joel on Software articles, mainly because I've read a lot of them and I can't be bothered to research the topics further! There'll be plenty of links to follow, plus the odd dead tree format recommendation.

Programming

This should be a no brainer. Lots of experience of programming in PHP, is not strictly necessary, a good programmer, particularly with experience of scripting languages or programming for the web should be able to pick up PHP in no time.

For someone who is basically a good software developer, learning another programming language is just not going to be a big deal. In two weeks they'll be pretty productive. In two years, you may need them to do something completely different in a programming language which hasn't even been invented.

- Sorting Resumes by Joel Spolsky

Most PHP applications are used in conjunction with an SQL database, predominantly MySQL, so you're going to need some of this under your belt.

Some knowledge of PHP is essential. Be aware of the benefits, the caveats and if you're interested, a little of PHPs history, some people really care about it. I think it definitely shows you are passionate about what you do or want to do. Maybe look to PHP's future, research whats coming up in PHP 5.3, or whatever the next version is at the current time.

Software Engineering

Most PHP roles go beyond just programming, so a good sense of what's involved in a full project life cycle should help you get that PHP job over the next guy. There are lots of processes and models available, but you don't need to be familiar with them all. Get a good idea of the 7 stages of the traditional Waterfall Model and you should be able to apply the principles to most methods. They are:

Requirements Specification
Design
Implementation
Integration
Testing
Installation
Maintenance

I like UML for design and documentation, so worth knowing about even if you haven't practiced it.

Libraries and Frameworks

If you are familiar with Object Oriented methodologies, arm yourself with the knowledge of PHP5's OO capabilities. Once you've got that, get a handle of the vast array of PHP frameworks that are available. You don't have to know them inside and out, just be aware of them and the benefits they give you. PEAR is a huge library of PHP code, check it out

Development Tools

There are plenty of tools available to aid and improve the development process, be familiar with as many as you can handle. I would insist on becoming familiar with, downloading and experimenting with subversion, or some other version control system.

Joel Spolsky has what he refers to The Joel Test. Later in this series, we'll discuss interviews, and I will recommend asking at least one of these questions at an interview, so you need to understand what they all mean and why they might benefit a software development team.

Security

Security is often a big cause for concern in the PHP world, mainly because it's not been handled correctly before. PHP is not insecure in itself, most vulnerabilities attributed to PHP are actually simply in softwares written in PHP.

Be aware of security issues in your code such as SQL Injection, XSS and CSRF. Also be aware of configuration directives that can affect the security of your PHP powered web servers.

Web Services

Understand what a web service is and some of the related technologies. PHP is ideal as a glue language, combining web services to consume single web services or create mash ups of several web services, but can also be used for providing web services.

System Administration

In my opinion, developers should be capable of administering the full stack they develop for, usually in this case, the LAMP stack. There can't be many potential PHP developers out there who don't have a spare computer or hard disk lying around that they can't install Debian on and follow a simple LAMP installation tutorial. If you've not got a spare hard disk, download VmWare Player and a debian appliance

I think thats all I can think of for now, I'm sure there's plenty I've missed. If there's any technical leads, managers or recruiters reading, please pipe up with what you expect from your applicants. The next part in the series will focus on the soft skills required for banking that PHP job.

Review: Deep Fried Bytes Podcast

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Tue, 12 Aug 2008 19:58:30 nil

[caption id="attachment_92" align="alignnone" width="300" caption="Deep Fried Bytes"][/caption]

As previously noted, I've been listening to a lot of podcasts recently, here's my first review

Deep Fried Bytes is an audio talk show with a Southern flavor hosted by technologists and developers Keith Elder and Chris Woodruff. The show discusses a wide range of topics including application development, operating systems and technology in general. Anything is fair game if it plugs into the wall or takes a battery.

The first thing I'll say about this podcast is that it's fun! The goofy intro music and southern accents are a breath of fresh air, especially for me being over on this side of the pond.

Woody and Keith never seem short of questions to ask and usually keep their guests engaged, often making them work pretty hard to explain their points. They make a point asking their guests to back track a little, getting them to give their slant on an acronym or buzz word that they have used.

Considering I'm an open source advocate, I've been pleasantly surprised to have enjoyed all the .net topics that have come up. While Deep Fried Bytes wants to be a Technology Podcast, as Keith and Woody are both currently .net guys, lots of their discussions and guests are based around Microsoft technologies. I think of the 9 episodes so far, only episode 4 with Joe Stump of Digg fame, has not involved some sort of Microsoft employee or MVP. This may be enough to put some non-Microsoft fans off, but I'd advise against it. Like I said, the shows are very interesting, good fun and might even give someone like me a little insight on what it's like on the Microsoft side of the fence.

Another thing worth mentioning, the podcasts usually have a sponsor, it currently plays for about 10 seconds part way through the cast, is fairly unobtrusive and isn't annoying or aggravating.

Conclusion

Definitely worth a subscription, doubt you will be disappointed. 8/10

Review: Gliffy.com

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Mon, 11 Aug 2008 11:23:06 nil

[caption id="attachment_72" align="alignleft" width="188" caption="Gliffy.com"][/caption]

I've been using gliffy.com for quite some time now and after recently signing up for a company licence, I thought I'd share my thoughts on it. This is far from a complete review, just some of the reasons we're making good use of it.

Gliffy.com is a free web-based diagram editor. Create and share flowcharts, network diagrams, floorplans, user interface designs and other drawings online.

Firstly, I've used many open source tools for drawing diagrams, particularly UML diagrams and they all have their drawbacks. Some of them do particular things really well, but all have ended up not quite cutting the mustard, although some of these projects may have come along way since I've used them. To name a few, I've used:

Diagram types

[caption id="attachment_80" align="alignnone" width="94" caption="UML Shapes"][/caption]

Next up, what do I need a diagram tool for? Most of the diagrams I draw are very simple. I draw small class diagrams, sequence diagrams, use case diagrams, simple flow charts and the odd network diagram. Gliffy provides a reasonable collection of shapes and icons for all of these diagram types, which you can simply drag and drop on to your canvas. The gliffy examples page includes a class diagram and a network diagram, both showing these predefined shapes.

Collaboration

At my workplace, we need to collaborate on diagrams. We have multiple platforms in use (2 linux, 4 windows), so whatever file format we use must be accessible to all developers. As a desirable, it would be nice if the format was none binary, so we can see the difference between revisions in the subversion repository.

[caption id="attachment_76" align="alignnone" width="300" caption="Gliffy Flowchart"][/caption]

Gliffy.com provides export to JPEG, PNG and SVG formats. For those not in the know, SVG is a standards based xml format for storing vector graphics. I tried the SVG export and opened the file up in Inkscape and was very impressed with the export. As you would hope, all objects in the diagram were editable, including the text. The actual diagram I exported had an embedded image in, so Gliffy exported the diagram as SVG and included it with the image in a zip file, which also worked great. Another benefit, SVG is an xml format, we can stick the exports in the Subversion repository and diffs work great. One dissapointing part of gliffy, is you can't upload SVG files to edit, so we have to stick with gliffy itself to make major changes, Inkscape for touch ups etc. This is probably a compatibility issue, the complete SVG specification is probably very difficult to implement.

[caption id="attachment_77" align="alignnone" width="260" caption="Gliffy diagram in Inkscape"][/caption]

File formats aside, gliffy provides shared storage for your diagrams, allowing colleagues to work on diagrams created by you. It also stores a revision history for you.

Usability

[caption id="attachment_79" align="alignnone" width="300" caption="Gliffy Edit Menu"][/caption]

Gliffy provides the typical editor capabilities, copy, paste, cut, delete and undo, so most people should feel right at home, though I aren't sure how well the keyboard shortcuts work. I use it with FireFox 3 on Ubuntu and the described shortcuts don't seem to work.

Aside from that, it's the regular run of the mill, drag and drop objects where you want them. It feels smooth enough, drop it in something like Prism and you probably wouldn't be able to tell it's a web app. One small gripe, if you have joined objects with the connector tool and then move the objects around, sometimes the connector flips it's angle.

Support

Worth mentioning, we've made two support requests and both were dealt with very promptly and efficiently. The first was creating a new billing account for our free trial under a different username and the second was one of my colleagues asking for some diagrams transferring from his personal account to his company one.

Value

At $25 a month for 10 users, Gliffy provides a excellent ROI for our company. For our six developers, Visio Standard would set us back a one off cost of 1079.88 GBP. 1079.88 GBP would buy us gliffy for 10 users, for just over 7 years. It's cross platform, accessible from anywhere where you have a web browser and an internet connection. Enough said.

Getting the right chair

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Sat, 09 Aug 2008 13:04:24 nil

Just over a week ago, I listened to Joel Spolsky and Jeff Atwood talking about chairs in their weekly stackoverflow podcast. Just yesterday, my manager did a sweep of our office asking if people are happy with their chairs, because one person had asked for a new one. My existing chair was actually broken so I asked for a replacement.

I previously read articles by both Jeff and Joel about equipping developers with a fancy chair and while I don't think they're wrong, having considered it, I still don't think I fit into that category. If I get uncomfortable at work, it's because I just spent 40 minutes playing football in my lunch break, don't want any more time to slip, so I skip getting a shower. Or I feel uncomfortable because we don't have air conditioning.

That said, even if I did decide I wanted a fancy chair, I doubt my company would see fit to buy me an aeron. Maybe if I'd been spoilt at some point, I'd come to realise the value of a quality chair, for now I'll just settle with the one my colleague is unhappy with.

A step in the right direction

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Tue, 15 Jul 2008 21:42:17 nil

At my current workplace, on a particular project we have not been allowed to use third party libraries, due to fears of licensing issues if and when we come to sell the product. The project is predominantly for the business, but it would be nice if we could convert the product into something we could market.

Despite multiple discussions and suggesting to management that open source software licensed under an MIT or BSD license would not have any implications, they would not have it. This has meant writing pretty much everything we need from scratch.

Recently, the managing director must have noticed the improved the user experience the little bit of JavaScript we use brings to the product. In previous conversations, I've mentioned to the MD that JavaScript is an area that the development team has little experience with, and this seems to have nudged him into letting us use a third party library or tool kit within the product. I spent a bit of time reviewing some of the frameworks and tool kits out there and basically boiled down to going with Dojo. Most of the frameworks reviewed provide similar features, Dojo came up on top because of the BSD license, recent announcement of a partnership with the Zend Framework and the added bonus of contributors having to sign a CLA. Looking forward to getting stuck into it and making our application a little easier on the eye.

The next task is to go about documenting the best way to include and use the tool kit in our application, will write about it if I get chance.

Zend PHP 5 Certified Engineer

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Tue, 15 Jul 2008 21:11:01 nil

After putting the exam off continuously for about 18 months, I actually got around to taking my Zend PHP 5 Certification exam. I originally scheduled the exam, put it off for a few months, then actually forgot about the exam altogether, only to reminded by google calendar an hour before I was due to take it. The test centre was at least an hour away, so that was a waste of $125. I scheduled the exam again, kept putting it off until I caved in last month and got it done.

The exam itself was considerably harder than the original, I wasn't all that confident when I clicked the 'End Exam' button, but I got the job done. I'm glad I found it harder, I've obviously improved my knowledge of PHP since I took the original exam nearly 3 years, so they must have made the certification more difficult to achieve.

As far preparation goes, I read the study guide right through once, making notes along the way. I turned up at the test centre about 20 minutes early and went through my notes. The study guide clearly did a good job!

<img src="http://davedevelopment.co.uk/wp-content/uploads/php5_zce_logo.gif" alt="Zend Certified Engineer" width="73" height="50" />

Processing output with SimpleXML

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Tue, 08 Jul 2008 11:13:36 nil

PHP's output buffering allows you to register callbacks, that will process the output before returning it back up the stack. This feature is usually used in conjunction with handlers added by PHP or it's extensions such as ob_gzhandler, which gzips the output, or ob_tidyhandler, which passes the output through tidy. As output buffers can be stacked, you can use many handlers in turn using the ob_start function.

<?php 

ob_start('ob_gzhandler');
ob_start('ob_tidyhandler');

?>
<!-- Your output here -->
</pre>

Because you can pass ob_start any function name you desire, adding custom handlers is easy. At my current workplace, we needed a quick fix for something we'd overlooked and because we'd (fairly) strictly stuck to generate decent markup, we used an output handler in combination with SimpleXml to manipulate the DOM in our HTML pages. I can't divulge the what we needed to do or the details of how we did it, but here's a basic example that will attempt to insert a CSS link to a page.

<?php 
function process_output($buffer)
{
    try {
        $xml = new SimpleXmlElement($buffer);
        if (isset($xml->head)) {
            $link = $xml->head->addChild('link');
            $link->addAttribute('rel', 'stylesheet');
            $link->addAttribute('href', 'http://o.aolcdn.com/dojo/1.0/dojo/resources/dojo.css');
            $link->addAttribute('type', 'text/css');
        }
        return $xml->asXML();
    } catch(Exception $e) {
        error_log($e->getMessage());
        return false;
    }
}

ob_start('process_output');

?>
<!-- Your output here -->

This code can be added to a file that's included in every page for a quick fix, but beware of performance hits. A quick test showed adding the above processing dropped this example files performance from 960 requests per second to 795 requests per second.

Fat Models and the Data Access Layer

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Tue, 17 Jun 2008 20:57:12 nil

There's been some discussion recently on why active record:

I like the active record pattern, so I don't think it sucks, but I do think it's used a little out of context sometimes.

If you're building a small lightweight app, then I think using your Data access layer as the M in MVC is a logical thing to do. It's quick, it's easy and you can extend either your active record in Rails, or extend your Table DataGateway in the Zend Framework and you wont go far wrong.

As soon as your app gets a little more complex, you might want to start creating custom models that contain more business logic than simply pulling and pushing to the database. If your application is complex enough, chances are your model will need to interact with more than one database table, if not database, so at this point, like Bill Karwin pointed out, your model should be using the DAL, not being the DAL. Loosening the coupling between model and DAL, should also help with automated testing the business logic, in that mock objects could replace the DAL.

The only problem is, I don't know the best way to do it.

I'm currently learning the ways of the Zend Framework and would be interested to see how people think the best way to implement this kind of complex model. I'm currently leaning toward something like this. I've included a Zend_Form object, to show how the Persons model encapsulates more logic than just pushing to and from the database. I think the biggest benefit of Zend_Form is validating input, which I consider domain logic, so should be part of the model. But I'm not sure the best way to make things easily testable, without pushing into the realms of fancy Dependency Injection and what not, which I'm not all that familiar with.

File: application/models/Persons.php

<?php

class Persons
{
    public function findByEmail($email)
    {
        $table  = self::getTable();
        $select = $table->select()
                        ->where('email = ?', $email);
        return $table->fetchAll($select);
    }

    public static function getTable()
    {
        // add some dependency injection?
        return new Persons_Table();
    }   

    public static function getForm()
    {
        // add some dependency injection?
        return new Persons_Form();
    }

    // ...
}

File: application/models/Persons/Table.php

<?php

class Persons_Table extends Zend_Db_Table
{
    protected $_name = 'persons';

    // ...
}

File: application/models/Persons/Form.php

<?php

class Persons_Form extends Zend_Form
{
    // ...
}

At first it may seem that the Persons model just ends up acting as a proxy to the Persons_Form and Persons_Table, but once you start writing methods that use both together, you'll start seeing fatter models and thin controllers, which is all good.

This really is a request for comments really, as I'm personally not sure about the best way to go about this. Would be interesting if any of the people using the MVC part of the Zend Framework in the real world go about this?

Software development podcasts

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Tue, 03 Jun 2008 22:30:38 nil

I recently picked myself up an 8GB Ipod Nano, I spend a fair bit of time walking my little dog Murphy and I've found it great for both music and more recently podcasts. So much so, that I quickly got few to the few podcasts I was subscribed to, so went in search of more. I've listened to and liked very much:

PHP Podcasts rss
Stack Overflow rss

The new editions to my list are:

Audible Ajax rss
Code Sermon rss
Google Developer Podcast rss
WebDevRadio rss
The Web 2.0 Show rss

I'll pipe back in a couple of weeks and give my thoughts on the new ones.

Switching server environments

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Mon, 26 May 2008 08:24:33 nil

After reading John Rockefeller's post on Handling multiple domains and less recently Richard Heye's post on displaying errors, I thought I'd write a little post about my qualms with their methods.

I won't go into too much detail, but both examples use a variable that can be manipulated by the user, $_SERVER['HTTP_HOST']. Richard actually changed his example to use $_SERVER['SERVER_NAME'], but as Chris Shiflett shows, neither are guaranteed to be genuine.

My example relies on having access to the server configuration, but is fairly simple. I think Ruby on Rails uses a similar method.

First we set up our virtual hosts, all pointing to the same codebase, but each getting an individual environment variable set using mod_env.

<VirtualHost *:80>
    ServerName davedevelopment.co.uk
    DocumentRoot /var/www/codebase
    SetEnv WEB_ENV davedevelopment.co.uk
</VirtualHost>

<VirtualHost *:80>
    ServerName test.davedevelopment.co.uk
    DocumentRoot /var/www/codebase
    SetEnv WEB_ENV test.davedevelopment.co.uk
</VirtualHost>

<VirtualHost *:80>
    ServerName anotherSite.com
    DocumentRoot /var/www/codebase
    SetEnv WEB_ENV another_site
</VirtualHost>

The code then switches on this variable, which should be guaranteed to be controlled by yourself?

<?php
switch($_SERVER['WEB_ENV']) {
    case 'davedevelopment.co.uk':
        $message = 'Welcome to DaveDevelopment';
        break;
    case 'another_site':
        $message = 'Welcome to another site';
        break;
    case 'test.davedevelopment.co.uk':
    default:
        $message = 'Welcome to DaveDevelopment Test';
        break;
}

echo $message;

?>

Being a Lead Developer - Part 3 - The Team

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Fri, 23 May 2008 21:46:18 nil

This mini-series is focusing on my experiences in my first Lead Developer role. Last time out I talked about the tools and methods I've introduced since taking on the role, this post is going to discuss any impact I've had on the team.

Laugh and Learn culture

Something I've tried to encourage within the team is to laugh at each others expense. This sounds bad, but I'm sure it helps lift morale and in turn we all learn from each others mistakes. It's not to say that we actively encourage mistakes, but everyone usually finds out when someone's written a WTF? piece and it may even make it to the wall of shame. I don't know if this will turn round and bite me in the arse because somebody complains about being bullied, but it seems to be okay so far.

Professionalism

With the new methods and tools we've recently introduced, I feel like we're starting to act like a proper development team and we're doing less of the gun slinging. I hope this is making people 'feel' like they are part of a proper development team, which should improve morale. On the flip side, I've been getting everyone to take the piss out of one another, which may be frowned upon, but I don't think that should be the case in a young industry.

Passion

This is something we particularly lack in the team, the developers get little creative freedom and tend to lack any sort of passion when developing.

"We need the developers thinking about their work when they're in the shower"

Basically, we're developing a system that will serve a great purpose for the company, but it's just not all that fun to develop. I've been lucky enough to take on some smaller projects, where I've had creative freedom, or I've gone ahead and created some projects of my own that have been useful as a learning tool and in the end provided the team with a useful tool. For example I wrote a simple daemon in ruby that watches log files, sending any output in bursts to a jabber chat room. I'm not saying work should be fun all the time, but a little freedom can keep the work force happy and greatly benefit the company

Being a Lead Developer - Part 2 - Software Tools and methods

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Mon, 19 May 2008 19:43:06 nil

This series is based on my experiences as a Lead Developer. The first part covered being a mentor, this part is going to focus on the tools and methods I introduced to the team in the last eight months.

Software tools and methods

A previous post on modern tools pretty much sums up the tools I've introduced to our team, the ones listed here are the ones that have been successful. To be honest, this sounds like a big list of buzz terms, tools and methods, but I can testify that these methods have helped improve our development environment.

Automated build, configuration and migration

We wanted a way to get the application up and running fairly swiftly in development environments and also a way of deploying code to testing, staging and production environments. I developed phing scripts to retrieve code from the subversion repository, configure it and then create/update the database. I feel this was successful, the scripts do what we need and as you'll see lead on to better things.

Continuous Integration

Once we had automated builds in place, it made sense to get some continuous integration going. Initially it was with vanilla CruiseControl, but later switched to phpundercontrol. I consider this a success from the developers point of few, particularly because mistakes made with database schema updates or procedures are quickly highlighted. On the other side, the managers haven't really embraced having a 'cutting edge' version of the application running and have pretty much ignored it. This may because of the dataset the database has, but that could easily be changed.

Unit Testing

This is definitely one of the most effective tools I've introduced and our collection of tests is growing. Originally I wrote a few unit tests for my own code using phpunit, but then started to encourage the other developers to have a go. One way in which I achieved this was to introduce the developers to writing tests for very easy classes/methods. We have alot of transfer object classes with various getters/setters, so to get them started I generated skeleton test classes for the value object classes and asked them to fill the gaps, even getting the trainees in on the act. Now everybody understands what the unit tests are for and how to go about writing them and we even uncovered a couple of mistakes in the transfer objects. Hopefully this will encourage the developers to not only write tests, but try to write code that can be tested in this way, along the low coupling, high cohesion track. Again, I see this as being a success with the developers, but will probably find managers questioning it's usefulness.

Scrums/Stand up meetings

Part of my job requires that I anticipate problems in meeting deadlines on the development plan, to deal with this I introduced daily stand up meetings. Initially people, myself included, found it hard to focus on the three basic questions...

What did I do yesterday?
What stopped me achieving all I wanted to?
What will I be doing today?

... but in the end we started to get going and I think everybody found them moderately useful. This went a little wayward when our little meeting room got taken over for a period of time, but then we started running the scrums in a chat room. Not only did this make it easier for everybody, we didn't all have to do it at the same time, but it stopped us getting carried away with things and also everything was logged. I can look back through the logs to see what we said at the scrum three Fridays ago etc.

Peer Code Reviews

This was quite a no brainer, it helps spot mistakes or code that's not quite up to standards. I basically wrote a ruby hook for the subversion repository that emails a random developer asking to run a code review on every fifth commit. I also involved the trainees, so they get to review code the full time developers are writing for the main system, giving them a little insight into the applications codebase. These have been successful and the team confirmed this in a recent meeting.

Scheduling and Estimates

I've tried to encourage the developers to generate and provide better estimates for their workload to the managers with mixed success. Despite trying to explain the benefits of using a simple estimating technique, only two developers have taken to it well. I'm happy for the others to generate their own tools and methods for providing estimates, as long as they work.

Being a Lead Developer - Part 1 - Being a mentor

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Sat, 17 May 2008 17:13:01 nil

I got promoted to my current position, Lead Developer, about eight months ago and I've recently been reflecting on what I think I've achieved and what I need to improve on, you can consider these posts thinking out loud. The role was newly created when I was appointed, we previously only had developer roles reporting straight to managers.

First, a little background on what we are and do. We are a small IT Department, working as a support team for a multi-disciplinary engineering contractor, including development of bespoke systems, mainly web applications. We're currently trying to move all these sub-applications into one big application. We currently employ three developers and two trainee developers. The developers work full time on the main application, while the trainees work through training material and small projects to gain experience.

Trainee Developers

When I first got promoted to the position, the main change to my daily work was being mentor to the trainees. When I started as Lead Developer, we didn't actually have any trainees, but were about to recruit. I was involved in the recruitment process, which was a good experience for me. I had actually asked my manager if I could sit in on a couple of interviews for experience before I got promoted, so I was interested in the aspects of hiring anyway. I did a little background reading on interview techniques [Recommended: 1, 2, 3, 4, 5], particularly regarding programmers and stuck to the technical side of things, allowing my managers to use their experience to judge the 'people skills'. Things went pretty well, but I'm concerned I probably wont ever do it enough to get good at it. Having said that, I'd rather keep the staff we've got than get really good at recruiting.

I inherited a training programme and modified it only slightly, except for the module on working as a team, which I rewrote. The trainees work through the programme completing exercises and taking on mini projects along the way. It seems to work pretty well, but I haven't given the actual programme any attention recently and should get it updated, to try and introduce some more modern development techniques.

The biggest problem I have, is not doing things for the trainees. Whenever they have a problem and ask for help, the temptation is to big up the keyboard and crank out the code they need. This would have benefits, the trainee gets decent code doing what they need and I can get on with my other responsibilities. The downside is, unless the trainee goes back and studies my code, they'll never learn. Instead I need to take my time and coax the answers from the trainee and let them solve their own problems, albeit with a little provoking

Recently I experimented with our newest trainee, who came to us with little actual programming experience. For his first project, I gave him the choice of implementing it the slightly harder way, which would take him longer but hopefully he would learn more, or the quick and easy way, leaving the harder stuff until later on. Thankfully he chose the hard way and I set him off on his way to use the Zend Framework. I like the idea of this, I know a lot people find it difficult moving from procedural techniques to object oriented methods, so why bother learning the procedural techniques first? I gave him a pointer by showing him Akra's superb tutorial and he went from there. On demoing the project to my manager, the trainee was asked "if the system was object oriented?" He replied "No". He didn't actually know he'd implemented the whole system, barring the view scripts and the boot loaders in object oriented PHP. At first I thought this was bad, thinking he hadn't realised what he was doing the whole time, but maybe it's just that he thought that was just the way things were done. I think this was a success and would be interested to hear how other companies introduce their trainees to frameworks and object oriented principles.

Part 2 - Software tools and methods

Log memory usage using declare and ticks in PHP

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Mon, 12 May 2008 21:38:06 nil

Update:See the first comment, looks a far better and more accurate way of tracking memory usage, thanks Mathieu.

As far as I know, there isn't any memory footprint profiling in Xdebug, I think there was at some point but they removed it because it was a little flaky. I like to monitor the memory usage within my scripts, and I've found this simple snippet can help. I don't think it's entirely accurate, but it can help a little. Unfortunately, I don't think it works all that well on Windows, if at all. All good on Linux though.

<?php

function log_memory() {
    $_SESSION['memory'][] = array(
        'time'   => microtime(true),
        'memory' => memory_get_usage(),
    ); // or insert your logging code here
}

declare(ticks = 50); // log every 50 ticks (low level instructions)
register_tick_function('log_memory');
$var = 'a';

for($i=0;$i<100;$i++) {
    $var .= 'a';
}

var_dump($_SESSION['memory']);

?>

PHP Versions in popular Linux Distributions

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Thu, 08 May 2008 19:33:40 nil

I had a problem today at work, I've been coding exclusively in PHP5.2 since it was available and most of the servers I've been working for are Debian or Ubuntu based, so I didn't have any problems until this afternoon. We've recently bought a SAN solution from Dell and to gain support we bought two new servers, both with SUSE Enterprise Linux installed, which only comes with PHP 5.1.2. That particular version came out in January 2006. Since then I've been using the new DateTime object, the filter functions, memory_get_peak_usage() and sys_get_temp_dir(). And they're only the problems I noticed. We could install from source, but then we lose the subtle benefits of package management.

So, this lead to me wondering what LAMP versions the popular distros are using, with the help of DistroWatch, I compiled this table. It only shows the community/open source distributions, the commercials counterparts for each are usually at least a year behind, guaranteeing support but only for out of date versions.

Distribution	Version	Apache	MySQL	PHP
Ubuntu	8.04 LTS Hardy Heron	2.2.8	5.0.51a	5.2.4
openSUSE	10.3	2.2.4	5.0.45	5.2.4
Fedora	8 Werewolf	2.2.6	5.0.45	5.2.4
Debian GNU/Linux	4.0 Etch	2.2.3	5.0.32	5.2.0
Mandriva Linux	2008.1	2.2.8	5.0.51a	5.2.5
Knoppix	5.3.1	2.2.8	5.0.51a	5.2.3
Slackware Linux	12.0	2.2.4	5.0.37	5.2.3
Gentoo Linux	2007.0	2.0.58	5.0.38	5.2.2
FreeBSD	7.0 RELEASE	2.2.6	5.0.45	5.2.5

Framework Popularity/Favouritism/Biase ...

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Mon, 28 Apr 2008 13:15:45 nil

Started writing this post over a week ago, kind of lost interest, but I polished it off...

After reading a couple of posts this week arguing the corner of both Ruby on Rails and Symfony, I found myself thinking about which framework I like the most. I came to the conclusion that I don't really have a favourite, just the one I'm currently using. I've used the two previously mentioned frameworks, CakePHP a long time ago and more recently the Zend Framework. I like them all. They all do similar things, sometimes in different ways, but having spent a lot of time maintaining and developing 'frameworkless' sites, I think they're all mega. Admittedly, my PHP is far stronger than my Ruby, so I steer towards the PHP ones, but that has nothing to do with the frameworks themselves.

Anyway, this lead me to have a little look on Google Trends, to see how many people are searching for what on the framework front, I included the previously mentioned four and Django, which is very popular, but my Python is worse than my Ruby, so I've not used it. I know this is hardly conclusive, but it was worth a look.

The results are pretty much what I expected. Django looks like it's a more common search term, but is starting to rise recently. Zend, CakePHP and Symfony are all slowly on the up, whereas Rails clearly had a big boom over the previous two years and is now settling down a bit.

How To: Simple database migrations with Phing and DbDeploy

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Mon, 14 Apr 2008 09:20:08 nil

Update 02/11/2011: I've since moved on from this method of running migrations, checkout phpmig on github.

Introduction

This How To will introduce some simple database migrations to your PHP application. Ruby on Rails is a popular web application framework, that provides a method of migrating (upgrading) the applications database programatically, keeping the database schema essentially version controlled. This allows individual developers to update their working databases and the databases on testing, staging or production machines to be updated with new versions of applications. The CakePHP framework has recently developed a migrations library simliar to rails, but this article focuses on using seperate tools to run database migrations, a build tool called Phing, along with a method for creating database migrations, dbdeploy.

Install Phing

I always use the beta or release candidate of phing and for the purposes of this article I suggest you do too. The best way to download and install phing is using PEAR. This can be done on Linux or Windows assuming you have the pear script in your PATH with three shell commands.

shell> pear channel-discover pear.phing.info
shell> pear config-set preferred_state beta
shell> pear install phing/phing

Example Application structure

As an example, we're going to develop a simple application with the following directory structure.

example/
 |-- db/
 |   `-- deltas/
 |-- deploy/
 |   `-- scripts/  
 |-- library/
 `-- public/

The db directory contains sql files for using and manipulating our database and the deploy directory contains our build scripts that set the migrations in motion. The library directory contains our application code and the public folder will contain scripts and files accessible directly from the web, but will not be the focus of this article.

Build scripts

This section shows you how to develop the build scripts that will run the database migrations. The first file we need to create is a simple configuration file and should be fairly self explanatory. The file is written as key=value, lines beginning with a # are comments. Open your editor and save the following text as deploy/build.properties.

# Property files contain key/value pairs
#key=value

# This dir must contain the local application
build.dir=../

# Credentials for the database migrations
db.host=localhost
db.user=user
db.pass=password
db.name=example

# paths to programs
progs.mysql=/usr/bin/mysql

The next file we are going to create is the deploy/build.xml file. This is the file that tells Phing what we want it to do. I'm not going to go into too much detail describing each part of the build file, there are some comments, but you should consult the Phing Documentation for further details and enhancements.

<?xml version="1.0" ?>
<project name="PurpleMonkey" basedir="." default="build">

    <!-- Sets the DSTAMP, TSTAMP and TODAY properties -->
    <tstamp/>

    <!-- Load our configuration -->
    <property file="./build.properties" />

    <!-- create our migration task -->
    <target name="migrate" description="Database Migrations">  

        <!-- load the dbdeploy task -->
        <taskdef name="dbdeploy" classname="phing.tasks.ext.dbdeploy.DbDeployTask"/>

        <!-- these two filenames will contain the generated SQL to do the deploy and roll it back-->
        <property name="build.dbdeploy.deployfile" value="deploy/scripts/deploy-${DSTAMP}${TSTAMP}.sql" />
        <property name="build.dbdeploy.undofile" value="deploy/scripts/undo-${DSTAMP}${TSTAMP}.sql" />

        <!-- generate the deployment scripts -->
        <dbdeploy 
            url="mysql:host=${db.host};dbname=${db.name}" 
            userid="${db.user}" 
            password="${db.pass}" 
            dir="${build.dir}/db/deltas" 
            outputfile="${build.dir}/${build.dbdeploy.deployfile}" 
            undooutputfile="${build.dir}/${build.dbdeploy.undofile}" />

        <!-- execute the SQL - Use mysql command line to avoid trouble with large files or many statements and PDO -->
        <exec
            command="${progs.mysql} -h${db.host} -u${db.user} -p${db.pass} ${db.name} &lt; ${build.dbdeploy.deployfile}"
            dir="${build.dir}"
            checkreturn="true" />
    </target>
</project>

That's essentially all the magic we need. Now we just need to create our database.

Writing dbdeploy delta scripts

We haven't actually created our database, so rather than create it the traditional way, we will actually use the migrations to create the initial schema. We've not actually decided what our example application does yet, but seeing as most tutorials make blogs, why don't we give that a bash. We'll start simple, one table with three columns called post.

Field	Type	Comment
title	VARCHAR(255)	The title of our post
time_created	DATETIME	The time we created our post
content	MEDIUMTEXT	The content of our post

Dbdeploy works by creating numbered delta files. Each delta files contains simple SQL to both deploy the change and roll it back. The basic layout of a delta file is like so.

--//

-- Run SQL to do the changes

--//@UNDO

-- RUN SQL to undo the changes

--//

We are creating our initial schema, so put the following content in db/deltas/1-create_initial_schema.sql

--//

CREATE TABLE `post` (
    `title` VARCHAR(255),
    `time_created` DATETIME,
    `content` MEDIUMTEXT
);

--//@UNDO

DROP TABLE `post`;

--//

Migrating the database

We are one step away from running our first migration. To keep track of the current version of the database, dbdeploy requires a table in the database. This is the only time we will have to interact with the mysql client directly.

shell> mysql -hlocalhost -uroot -ppassword example
mysql> CREATE TABLE changelog (
  change_number BIGINT NOT NULL,
  delta_set VARCHAR(10) NOT NULL,
  start_dt TIMESTAMP NOT NULL,
  complete_dt TIMESTAMP NULL,
  applied_by VARCHAR(100) NOT NULL,
  description VARCHAR(500) NOT NULL
);
mysql> ALTER TABLE changelog ADD CONSTRAINT Pkchangelog PRIMARY KEY (change_number, delta_set);

We are now ready to run our first migration and create the initial schema for our application.

shell>cd deploy
shell>phing migrate

All being well, we now have a posts table in our database. But what about an author for our blog posts? We'll have to add another table and a foreign key from the post table to author table. To do this we create another delta, we call this one db/deltas/2-create_author_and_link_to_post.sql

--//

CREATE TABLE `author` (
    `author_id` INT(10) unsigned auto_increment,
    `name` VARCHAR(255),
    PRIMARY KEY (`author_id`)
);

ALTER TABLE `post` ADD `author_id` INT(10) unsigned NULL;

--//@UNDO

ALTER TABLE `post` DROP `author_id`;

DROP TABLE `author`;

--//

Run our migrations again.

shell> cd deploy
shell> phing migrate

Conclusion

That's pretty much it, you've seen how to create database deltas and use them to migrate your database, if you can't be bothered to copy and paste things to try for yourself, download the example application.

There are plenty of caveats when it comes to version controlling databases, especially if you branch and merge your application code, some are detailed in the dbdeploy documentation

This tutorial is probably incomplete or wrong in plenty of ways, if you think you have something to point out, please leave your comments below

Contributing to Open Source Software

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Thu, 03 Apr 2008 16:09:56 nil

I'm a big fan of Open Source Software. Although I'm sure I could get by if I had to move to proprietary based software, at the present time, open source software earns me a living. Because of this, at the start of the year, I decided to give something back and start contributing to open source. I chose the Zend Framework, because it is written in PHP and is a project I have a lot of interest in. I registered for an account on the bug tracker and signed the CLA, but that's as far as I got. I started looking through some of the bugs available for fixing, but found that the simpler ones were being dealt with quickly by other people and the only ones left where a little too complex for me. I still plan to contribute before the year is over. I intend to use the framework for a project I have in mind and the frameworks current implementation of a client for Amazon Web Services needs adding to for the purposes I need, so maybe I can contribute that way.

Back to the point, today I think I found a bug in PHPUnit, I couldn't work out how to submit a bug report via the website, so I emailed Sebastian Bergmann with a patch. I'm not going to describe the bug until I know for sure that I'm correct, but I'm at least 90% sure I'm in the right. So, if I am right, that's my first solid contribution to open source software.

Update

Well it seems I was right in reporting the bug, but it appears Sebastian didn't like the patch I sent, or more likely it didn't work. This changeset looks slightly different to the patch I submitted.

Index: PHPUnit/Util/Metrics/Function.php
===================================================================
--- PHPUnit/Util/Metrics/Function.php   (revision 2707)
+++ PHPUnit/Util/Metrics/Function.php   (working copy)
@@ -449,7 +449,7 @@
         }

         else {
-            $this->crap = pow($this->ccn, 2) * (pow(1 - $this->coverage/100, 3) + $this->ccn);
+            $this->crap = pow($this->ccn, 2) * pow(1 - $this->coverage/100, 3) + $this->ccn;
         }
     }

From what I can tell, the change put into place is also incorrect according to the C.R.A.P. index formula.

Update 2

I emailed Sebastian and he fixed it properly in changeset 2712

Linux/PHP one liner - Syntax check all files.

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Tue, 25 Mar 2008 17:41:21 nil

Here's a simple one liner you can use to syntax check all php files in your working directory.

find . -type f -name "*.php" -exec php -l {} \; | grep -v 'No syntax errors'

For those not familiar with the programs used, it basically reads as.... Find all files that end in '.php' and with each of those files run php -l. This is then put through a pipe to the grep -v, which filters out all files that are syntactically correct.

Sifr turned off

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Thu, 20 Mar 2008 19:24:09 nil

I noticed some problems with my browser (FF2/Ubuntu) with the Sifr headings on the posts, so I've commented them out, sorry about the horrible blue heading links ;)

10 tools for Modern PHP Development

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Thu, 20 Mar 2008 19:16:35 nil

A simple list of tools for modern PHP development. There are alternatives to most of the tools, but I'll list native PHP tools wherever possible.

1. PHPUnit

PHPUnit is a testing framework belonging to the xUnit family of testing frameworks. Use it to write and run automated tests. Start using PHPUnit

2. Selenium RC

Selenium RC can be used in conjunction with PHPUnit to create and run automated tests within a web browser. It allows tests to be run on several modern browsers and is implemented in Java, making it available to different platforms. PHPUnit and Selenium

3. PHP CodeSniffer

PHP CodeSniffer is a PHP code tokenizer, that will analyse your code and report errors and warnings based on a set of Coding Standards. Documentation

4. Phing

Phing is a project build tool and is a PHP port of the popular Java program ant. Phing can be used to automate builds, database migration, deployment and configuration of code. Documentation. Database migrations with phing

5. Xdebug

Xdebug is a multi-purpose tool, providing remote debugging, stack traces, function traces, profiling and code coverage analysis. Debug clients are available in many PHP IDEs and even plugins so you can debug from everybody's favourite editor vim. Documentation

6. PHPDocumentor

PHPDocumentor is an automated documentation tool, that allows you to write specifically formatted comments in your code, that can be brought together to created API documentation. Tutorial

7. phpUnderControl

phpUnderControl is a patch for the popular Continuous Integration tool, CruiseControl. Together with the previous six tools, phpUnderControl gives you a great overview of the current state of your application/codebase. Keep an eye out for Xinc

8. Zend Framework - or <insert your favourite framework here>

Frameworks facilitate the development of software, by allowing developers to focus on the business requirements of the software, rather than the repetitive and tedious elements of development, such as caching. There are plenty of frameworks to choose from, but I particularly like the Zend Framework. Have a read through this excellent Getting started guide

9. Subversion

Subversion is a revision control system that has superceded CVS. If you're writing software of any kind, you shoud be using version control software.SVN Book

10. Jira

So I could have named one of many, but this is the one I've liked the most recently. Jira is a bug/issue tracking software package and can also help with project management in terms of goals and roadmaps. Most issue trackers link to version control repositories, such as Subversion. Only downside to Jira is that it costs for non open source projects.

I'm pleased to say that with a little bit of pushing and persuasion by myself, we are currently using all of these technologies with the exception of Jira, we have a bespoke issue tracker.

What do you think to the list? Anything I have missed? Any alternatives you prefer?

New site - LimoTrack.co.uk

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Sun, 23 Sep 2007 04:06:18 nil

Kind of pre-launched a site for a friend this week, he's started a new business with a slight twist of a limosine service, he's going to be ferrying people around in a fully-tracked articulated vehicle, which is kind of like a tank people carrier. Will post later with a press release and a few technical tidbits on the site.

Hardman LimoTrack

Capistrano, Migrations and Which-Broadband updates

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Tue, 11 Sep 2007 00:07:43 nil

I've been sitting on some code updates for www.which-broadband.net for a while now, what seems like ages ago I added a simple news section and adding link redirecting along with click tracking, but didn't get around to releasing it. I actually made a couple of sales of the site the last couple of months, so I thought I'd make a little effort to tidy things up, but decided to do it the long winded way and learn something in the process. I've read lots about Capistrano before, but never really put it into practice, now seemed like the time and also to try the ActiveRecord's migrations. Most of it went swimmingly, the biggest annoyance I found was ActiveRecords MySQL implementation, in that the BIGINT's I had were only integers to ActiveRecord so once I created the database with migrations, the columns wouldn't hold a number large enough. I got round this by making the columns strings and overiding the accessor and modifier in the model in question and then things were fine. I also had problems where I was reading the Capistrano one manual, but using version 2, which has some major differences. Anyway now I have an updated version of my site online, plus an automated deployment tool readily available, lets hope I'll make more frequent updates. I'm doing a few little tests with AdWords to see if I can do any click-flipping, if nothing works I'll just leave it alone for now, don't have the time to push it too hard.

Getting things done

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Tue, 10 Jul 2007 09:21:51 nil

Been a long time since I've posted, taken a step back from freelance work as of late and I've been doing lots of other things. One of them is reading David Allen's Getting Things Done. Thus far I've found it pretty interesting and just knowing that I should organise myself better has lead to some improvements. I've started using some software to keep track of things, it's a Ruby on Rails application based on the principles of Getting Things Done, I just run it locally using webrick and it's quite nice, definitely worth a look.

How To: Send SQL commands to a database in VIM

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Thu, 29 Mar 2007 09:02:30 nil

A simple way of sending SQL to your database from everyone's favourite editor Vim, without any plugins or macros.

Firstly we need to add a custom command to our .vimrc, open it up and add the following line, where:

CommandName - The name of your command
Username - The username for this database
Password - The password for this database
Database - The name of the database

:command -range=% CommandName :<line1>,<line2>w !mysql -uUsername -pPassword Database -t

For example:

:command -range=% SendDB :<line1>,<line2>w !mysql -utest -ptest test -t

That's all the setting up we need. As you can see, after the exclamation mark is just the regular MySQL Command line tool and various options.

Fire up vim and start editing your sql.

# vim test.sql

As an example, here's some simple SQL.

-- Drop existing table
DROP TABLE IF EXISTS `test`;

-- Create table
CREATE TABLE IF NOT EXISTS `test` (
    `test_id` INT(11) NOT NULL auto_increment,
    `name` VARCHAR(255) NOT NULL,
    PRIMARY KEY (`test_id`)
);

-- Add our data
INSERT INTO `test`(`name`) VALUES('dave');

-- Get it back out again
SELECT * FROM `test`;

If we're happy with what we've written, while in command mode, type :SendDB, or whatever you called your command and hit enter. If it works, you'll see something like this.

~/test.sql[+][sql] unix 
:SendDB
+---------+------+
| test_id | name |
+---------+------+
|       1 | dave | 
+---------+------+

Press ENTER or type command to continue

You can always add to the custom command, in particular piping the output to less can be effective for larger result sets.

Get paid to search the web with SlashMySearch

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Fri, 16 Feb 2007 09:33:01 nil

SlashMySearch is a internet search engine that pays it uses as they search. An interesting thought, I'm sure there are plenty of sites like this out there, but I thought I'd give it a try. I came across it at CJCM and IT via ProBlogger.

Simply signup and set your unique URL as your homepage. I don't think it's everybodies cup of tea and it might get a little annoying but it's got to be worth a try. They also run a referral program, so tell your friends!.

DaveWP Wordpress Theme

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Tue, 06 Feb 2007 18:26:33 nil

I figure it's about time I updated my blog and designed a new theme, but before I do I thought I would tidy this one up and release it. just like I told 'Big John' from Position is Everything I would do well over a year ago. It's far from perfect, but I kind of like it and it made a good project for me to experiment with some new design technologies.

The layout is based on the The Jello Mold Piefecta Layout, being a 3 column, source ordered, fixed width, equal height column layout. The template also uses sIfr ( to produce rich and accessable headings.

Looking back I was clearly influenced by Mike Davidsons Blog design, for which I apologise to him and in no way take credit for the 'look' of the theme.

Get the zip or gzipped tarball. Feedback and comments would be appreciated, but not quite as much as bug reports and fixes.

PHP Conference London 2007

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Tue, 23 Jan 2007 06:36:03 nil

A couple of days behind with this one, but I thought I'd post to help out. As alot of the PHP community have mentioned, The PHP London user group are running the UK's second dedicated PHP conference on Friday, 23rd February. Confirmed speakers thus far are, Cal Evans, Simon Laws, Kevlin Henney and Rasmus Lerdorf.

I didn't make it last year but will see what I can sort out this year.

Joined Elance.com

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Mon, 22 Jan 2007 16:23:46 nil

After reading this news article, (funnily enough through my RSS feed of digg.com's programming section) I decided to sign up for a month to see what Elance.com was like.

Basically Elance is a community to bring together people who want to do work and people need work doing. People or companies post projects in the various categories and then service providers must make proposals, the most impressive being likely to get the contract. Posting projects is free and the charges are made to the people providing the work at a rate of 8.75%.

This seems quite costly for the freelancers, expecially seeing as you have to pay to subscribe as well, but as a trial I'll try and complete projects at cost and not make a profit.

Desktop switch to Ubuntu

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Tue, 19 Dec 2006 18:19:06 nil

Just a quick note, I've recently moved from Gentoo to Ubuntu on my desktop machine at home. I had a few problems with my Gentoo install and while I find Gentoo great for learning the ins and outs of things, as I do more and more freelance work, I need a OS environment that's not going to let me down. My point being environment, Gentoo is more than capable of being extremely stable, but it takes a lot more knowledge to get it and keep it stable. Whilst I was happy learning and taking my time before, things need to happen a little more urgently now.

I chose Ubuntu mainly because I run Debian on my dedicated servers and am already familiar with the package management, but also because I wanted to see what the fuss is a bout. I'm very pleased so far. I'm running Mozilla Firefox, Mozilla Thunderbird, GNUCash, OpenOffice.org, Amarok and KTorrent day to day, the latter two despite using GNOME for my desktop.

Projects in the pipeline

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Fri, 08 Dec 2006 11:46:09 nil

I have a few projects that are in the pipeline, some are at the design stage, some are just notes written in my 'ideas' pad, thought I'd share a few.

Ingleton Plumbing and Heating

This is a project I started about 4 years ago for a friend, at one stage I got a fairly decent design done and was just waiting on a bit of content, but that got lost along with the design for the ATST Solutions site after a hard drive failure and a lack of backup. I've got a very basic design going, needs some flair.

PHP Book Reviews and Star Wars Book Reviews

I decided to start these two as little niche book review sites. I read a lot of Star Wars books and a few PHP books and figured I could write my own reviews, use amazon's web services to get other peoples views and attempt to make money through the affiliate links. I decided to do this seeing as one of the most popular posts on this blog is the Essential PHP Security Book Review.

A price comparison site

I have a few ideas aimed around creating a price comparison site, starting small but hopefully getting big. I plan on using product feeds from the various affiliate networks and have a rather cunning marketing plan, not one that hasn't been done before, but I don't think anybody has done it in this area before. I've got a couple of thoughts on the name, one of which is treading a little too closely on PriceRunner, but I'm in no rush with this one.

Google taking over day to day tasks...

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Thu, 07 Dec 2006 18:21:30 nil

I'm sorry to say it but I've started to cave in and use Google's range of web applications to deal with day to day tasks. It all started when I finally made myself start using a calendar and I figured Google Calendar would be the best place to start. Since then I've started using the Alerts service, sending emails to the GMail account I've had for ages but never used and now I've even switched from Bloglines to Google Reader. I've also been using Google Analytics to track statistics on the DaveProxy site.

The following is a little review of each application, what I like and what I don't.

Google Calendar

Google Calendar is a shared calendar application, allowing users to manage their personal calendar and publish parts they would like to, to colleagues and friends. There are also other shared calendars, for example I have Barnsley's fixtures showing in my calendar. I've no complaints to make about the calendar application, it's nice and easy to use and it sends me SMS reminders, all free of charge.

Google Alerts

Whilst I haven't used the Alerts service all that much, I think it's an awesome tool. It's extremely easy tool to use, I was quite suprised to see my Boss using it, receiving alerts based on our company name. Basically you register keywords with the alerts service and tell it how often you want emailing. If any new pages turn up in the google index within that timeframe, you'll receive an email with a link to that page. Simple.

GMail

Again, I've not used gmail too much, right now I only receive my google alerts to my gmail address, but as far as webmail goes it's pretty good. I'd definitely like to see the ability to drag and drop like a RoundCube installation can, but otherwise it's more than what I need.

Google Reader

I've been using Google Reader to read and manage all my RSS feeds for about a week now after switching over from Bloglines and I have mixed feelings. Importing my feeds was easy enough, thanks partially to Bloglines being nice enough to allow me to export them. Google reader by default marks feed entries as read when you scroll past them, which seems pretty cool but two things annoy me. The first is probably a browser issue, but whenever an entry is marked as read, the number in brackets next to the feed or category link on the left changes, as it should, but this causes some of my category links to span two lines, then one line, then two lines again. It's basically a little flicker on the left hand side of the screen which is a little distracting. The second thing is, if there's only one entry in a feed or category that is less than the windows height in length, I don't scroll, so it doesn't get marked as read. Other than that, it could do with being a little faster, but I'm sure it'll be constantly improving on that front.

Google Analytics

I'm very impressed with the analytics service, it makes it very easy for me to monitor the proxy website, which AWStats isn't really appropriate, definitely worth checking out.

New Site Launched: Pontiac Trans Am UK

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Tue, 21 Nov 2006 16:54:30 nil

This past weekend I acquired myself a new baby and decided to put a site up about it. It's all very basic and rushed, running off Wordpress and ZenPhoto, but cool all the same.

PTAUK

Quick Tip: Cygwin here shortcut, Explorer here shortcut

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Thu, 19 Oct 2006 04:53:50 nil

Thanks to this mailing list post I now have a 'Bash here' option in my windows explorer context menu for folders and drives. Simply add this registry script to the windows registry.

Thanks to my self and this little script, I can open a 'windows explorer window here' from within cygwin.

#! /bin/bash
PWD=pwd
PATH=cygpath -w $PWD
/cygdrive/c/WINDOWS/explorer.exe $PATH

New site launched for AJW ltd.

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Wed, 18 Oct 2006 06:11:49 nil

I launched a new site today, assuming the updated DNS records are rippling around the world as I type. The site was done as a favour to a friend, is very basic but I think it gets the point across.

The company are a bespoke joinery specialist, particularly architectural joinery and trade as Architectural Joinery Workshops Ltd. (AJW)

Still a few things to do to the site, but I'll get those done shortly. Namely, some sort of templating system to ease updates and modifications, meta tags and then a little bit of marketing to see if we can develop some decent rankings.

Congratulations to the BareNakedApp guys

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Thu, 07 Sep 2006 03:57:49 nil

I just wanted to say congratulations to Ryan and his team and to thank them for running the blog, it's been quite inspirational and extremely informative for me. One day, once I come up with the right idea, I might follow in their footsteps.

If you didn't know, for a while now, Carson Systems have been developing their new web application, Amigo, and it launched a few days ago. The great thing about this is they have shared their progress through a blog, BareNakedApp. I know it's a little late to be telling people about this, but you can have a quick look through their archives.

Press Release: Broadband website launch

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Wed, 06 Sep 2006 12:09:33 nil

FOR IMMEDIATE RELEASE:

Looking for the right broadband deal? Look no further...

ATST Solutions launches http://www.whichbroadband.net

Today marked the launch of ATST Solutions new website, which-broadband.net. The website intends to fill a growing need for UK consumers looking for the right broadband deal in a single place.

Which-Broadband.net is a simple broadband comparison site, listing broadband deals with the most important factors for making your decisions. The site avoids confusing garble, leaving readers with the straight facts.

Whilst not unique, ATST Solutions owner Dave Marshall says: "There's plenty of competition for sites of these kind, but most are out of date and misleading to the public. Which-Broadband intends to tie in closely with just the top few providers to keep track of the latest offers."

Which-broadband.net lists broadband packages from 7 of the largest internet service providers in the country in a simple 'at a glance' format, to take a look, please vist http://www.whichbroadband.net.

Notes to editors:

ATST Solutions is a small startup company based in the UK, who aim to deliver experise in a broad range of web development and marketing.

For further information contact: Dave Marshall +447740346652 pr at atstsolutions dot co dot uk

Quick bandwidth optimisation tips for Apache and PHP

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Mon, 04 Sep 2006 11:25:55 nil

Here's a quick tip to optimise the HTTP response headers sent by your webserver. While being pretty useless to the average user, the Server signature and powered-by headers could be removed or at least reduced. Obviously these changes are only minor, but on a heavily loaded server such as large forums, could make a decent little saving.

These two little changes...

apache2.conf

ServerTokens Prod

php.ini

expose_php = Off

Will change the HTTP response headers on this server from:

Server: Apache/2.0.54 (Debian GNU/Linux) PHP/4.3.10-16 \
mod_ssl/2.0.54 OpenSSL/0.9.7e mod_perl/1.999.21 Perl/v5.8.4
X-Powered-By: PHP/4.3.10-16

To:

Server: Apache

That's a saving of 128 bytes. This site has served 2025 requests so far this month, so this little trick might only have saved me 253kB, but what if there had been 20,000,000 requests?

Ruby on Rails - First Impressions

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Mon, 28 Aug 2006 11:29:15 nil

I've finally gotten around to finishing the basic design on the Broadband affiliate site project I've had ongoing for a while and figured it would be nice to have a backend to add/remove, de-activate all the different packages etc. So, I decided to go crazy and have a look at Ruby on Rails.

Now I'm not extremely well versed with patterns and so on, but I have a decent understanding of the MVC style architecture that seems to be popular these days and I've seen a few tutorials on Symfony, a PHP 5 web framework which I believe is fairly similar in nature to Rails, so I figured I'd get along okay. I feel I was pretty right, in a few hours over today and yesterday I've come out with a working database driven website and administration area running on Ruby on Rails.

Running Gentoo, I always try and use the portage package management tool, to be double sure on what I needed to do I had a little search on google which led me to this page, and these commands.

echo "dev-ruby/rails mysql fastcgi" >> /etc/portage/package.use
emerge -av rails
rails /path/to/app

That all went well and I was sat looking at a clean rails application. I fired up the builtin webserver, WEBrick, it worked, so I closed it again. Seeing as I use Apache on all my servers, I wanted to use it for the development aswell. I've never used FastCGI before, but what do you know there was an example virtual host definition in the rails README file.

  
  <VirtualHost *:80>
    ServerName rails
    DocumentRoot /path/application/public/
    ErrorLog /path/application/log/server.log

    <Directory /path/application/public/>
      Options ExecCGI FollowSymLinks
      AllowOverride all
      Allow from all
      Order allow,deny
    </Directory>
  </VirtualHost>

Once again, this worked a treat. I then set about creating my application, which by enlarge went along without many flaws, using a combination of the excellent Rails API, this Create a weblog in 15 minutes screencast and Rolling with Ruby on Rails as guidance. The biggest problem I had was creating initially trying to create the scaffolds. Not knowing RoR's naming conventions, I'd foolishly named my database tables in the singular form, ie provider rather than providers. After overcoming this, I rolled on pretty nicely.

Using this page as guidance I used the login_generator to create a simple admin area, and the scaffolding stuff filled most of the admin pages for me, just a few tweaks here and there were required. The public face was even easier, basically allowing a few different ways to filter the list of broadband packages through one action, 'list'.

As far as the language Ruby itself goes, I've barely learnt anything, basically because Rails does it all for me. The most complicated things I did code wise, was using a case statement to change the filtering on the public packages page and this simple function for turning bytes into a more readable form.

        def human_readable(number)
                count = 0
                while number/1024 > 1
                        number = number/1024
                        count += 1
                end

                iec = ['', 'K', 'M', 'G', 'T']
                return number.to_s + iec[count]
        end

My only gripe with this so far, is the speed. It does take forever to generate these simple pages, the built in webserver does seem to be a shade quicker, but I'd still rather use apache. I'm sure there'll be some tweaks I can make to speed the FastCGI module up, but there's no rush for that now.

Hopefully I'll signup for a few affiliate programs and launch the site properly within the next few days, I don't expect to make a fortune but it's been a good little learning utility so far and I hope to use it to learn a few things about affiliate marketing.

Beautify command line CVS output with perl

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Tue, 15 Aug 2006 03:35:10 nil

Here's a couple of little perl scripts I use on a day to day basis, they simply beautify the output of command line CVS client program. I'm not a perl programmer so I'm sure there are much better ways of doing this, any suggestions are welcome. I also haven't tested either of them on anything but cygwin/winXP, cvs 1.11.17 and perl 5.8.7.

file: cvs-update

This little script simply takes the output from cvs update and removes most of the useless information and colours the remaining output, leaving something like this.

U docs/README
P docs/INSTALLM
M docs/FAQ
C docs/TODO

file: cvs-editors

This script takes the output of cvs editors and formats the output neatly in an ascii table, like this.

+--------------+---------+--------------------------+---------+
| File         | User    | Date                     | Host    |
+--------------+---------+--------------------------+---------+
| docs/README  | davem   | Mon Aug 14 10:04:56 2006 | lando   |
+--------------+---------+--------------------------+---------+
| docs/FAQ     | davem   | Mon Jul 31 14:38:12 2006 | lando   |
+--------------+---------+--------------------------+---------+
| docs/TODO    | davem   | Mon Jul 31 14:34:56 2006 | lando   |
+--------------+---------+--------------------------+---------+
| docs/INSTALL | davem   | Mon Jul 31 14:04:31 2006 | hansolo |
+--------------+---------+--------------------------+---------+

New domains for DaveProxy

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Mon, 14 Aug 2006 02:07:36 nil

I registered four new domains to point to DaveProxy, might help some folk who have been blocked.

Upgrade to WordPress 2.04

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Thu, 03 Aug 2006 14:19:44 nil

That's right I finally bit the bullit and upgraded this baby to WordPress 2.04. I stayed away from the 2 release at first, just to let it get smoothed out and then just got lazy. It didn't take ten minutes, the upgrade instructions on the site being slightly overkill, mainly backing up everything, but better safe than sorry.

The best thing to come out of this is I'm now using the Akismet spam blocking plugin and have been very impressed with it. Since activating it on Sunday afternoon, it's captured 160 spam comments, and successfully let the 5 genuine comments through.

You'll also notice the related posts links beneath all posts and you wont notice but I'm running the Google Sitemaps Generator aswell.

Identity Package Contest Winner

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Thu, 27 Jul 2006 02:05:50 nil

Announced the winner of the ATST Identity contest on Sunday, finally got my debit card verified on Money Bookers today and the transaction is complete.

The winner was GeniusLogo, with this logo and equivalent package.

Other worthy mentions were efforts from:

LogoHolik with this entry
Chris318 with this entry
ZidaNe with this entry
Casperamy with this entry
Huhknee with this entry

Brainbench PHP 5 Certification

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Sat, 22 Jul 2006 05:16:14 nil

Intrigued by reading about the BrainBench PHP certifiation on Tobias Schlitt's and Markus Wolff's respective blogs (via PlanetPHP), I thought I'd give it a try.

To basically iterate what they both said, it's bollocks. You've got three minutes to answer each question and as long as you're quick with the online manual, most of the answers can be looked up easily. That said I didn't get them all right, so I must have missed with a couple of guesses for one's I couldn't lookup. One in particluar was a function supposedly for verifying credit card numbers, the logic was too much for me to take in, not in three minutes anyway.

The code examples are filthy as well, the first one I got involved two ternary operators spread over two or three lines and similar to the Zend Certification, I think syntax highlighting would be nice on these things.

I do think being able to use the manual well is valuable skill, however. I have the PHP.net Manual set up as a quick search in Firefox, so an average search is simply a case of hitting ctrl-t (new tab), alt-d (focus to address bar), php function_name, enter. I do this all day long, I'm not bad at coding, but struggle to remember parameter order etc.

Here's my results anyway.

Company Identity Package Contest

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Sun, 16 Jul 2006 13:48:52 nil

Right then, I've finally bit the bullet and created an identity package contest over at SitePoint. It's a contest to design a logo, letterhead and business cards for ATST Solutions which is my emerging business as a sole trader, mainly for the little bit of cash I'm making from my advertising on various web sites, but I've started a small web design contract for a friend, more on which later.

I decided to do it this way for a number of reasons:

Although technically capable with some aspects of design, I lack creativity and any artistic qualities.
The excellent quality of works in the other SitePoint contests
This way lots of people get to throw ideas at me and I pick the one's I like, eventually picking the one I like most.
I wouldn't know where to start commisioning someone to do it in a none contest sense.
I think the money spent on SitePoint listings and prize money is good value for the quality you get.

Will post the winning entry when I get it, the contest ends this Friday. I'll also post links to any designers websites who made good impressions and came in close.

PHP5 objects assigned by reference?

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Sun, 18 Jun 2006 03:46:51 nil

I've been working with PHP5 since it's release and find it very hard to believe I haven't come across this yet. By default, in PHP5, assigning an object instance to another variable, the new variable will access the same instance, but not by reference? To quote the manual:

When assigning an already created instance of an object to a new variable, the new variable will access the same instance as the object that was assigned. This behaviour is the same when passing instances to a function. A new instance of an already created object can be made by cloning it.

Hence the code... <?php

class MyClass { var $myVariable = '1'; }

$instance1 = new MyClass(); $assignNormally = $instance1; $assignByReference = &$instance1; $clone = clone($instance1);

$assignNormally->myVariable = '2'; $clone->myVariable = 'clone';

var_dump($instance1); // 2 var_dump($assignNormally); // 2 var_dump($assignByReference); // 2 var_dump($clone); // clone

$instance1 = null;

var_dump($instance1); // NULL var_dump($assignNormally); // 2 var_dump($assignByReference); // NULL var_dump($clone); // clone

Produces ...

object(MyClass)#1 (1) { ["myVariable"]=> string(1) "2" } object(MyClass)#1 (1) { ["myVariable"]=> string(1) "2" } object(MyClass)#1 (1) { ["myVariable"]=> string(1) "2" } object(MyClass)#2 (1) { ["myVariable"]=> string(5) "clone" } NULL object(MyClass)#1 (1) { ["myVariable"]=> string(1) "2" } NULL object(MyClass)#2 (1) { ["myVariable"]=> string(5) "clone" }Had me puzzled at work for quite some time and I ended up searching the Bug reports.

DaveProxy 3.0

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Fri, 26 May 2006 08:43:27 nil

Well then, a few months down the road and I've hopefully made a few more improvements to DaveProxy.

First off is some modifications to the CGIProxy script, thanks to the guys over at Proxy Society. It basically allows me to control which type of files can be accessed through the proxy, based on the file extension.

I did have Squid running as a httpd accelerator for a while, but in the end I decided that the benefits were not really worth the hassle. On the other hand, I can see where this type of setup could prove very useful, particularly with other CGI scripts, just not proxy one's.

I also started rotating the cgi-bin folder by adding a random number to the end, changing all the configuration scripts and the index page of the site every two hours. This certainly stopped a lot of hotlinking, but stopped people from adding favourites through the site and also any incoming traffic from Proxy.org. I also tried using apache to deny any requests without referers, this worked well for a short time...

The site had some major downtime recently and although I'm not entirely sure I think I figured out what happened. About three days ago, I had a massive influx of traffic, which ended with me serving thousands of 403 denied requests, which culminated in the proxy not being able to handle any real requests because apache and the proxy itself were firing that many 403's back. I personally think these requests were mainly coming from China. Pages like this providing links to FreeBSD and not to mention all the forum pages hotlinking porn, are just abusing the site and it's not fair on me or the genuine users. First I thought about geotargetting and deny requests, but that wouldn't help anyway. A 403 might cost me 1kB, which doesn't sound like much, but times that by thousands and my server would be crippled again. So why not drop it before it gets to Apache? A quick google search not only led me to sites listing chinese and korean 'A' class network ranges, but also to iptables scripts set to drop all traffic for them. Changed the port from SMTP to HTTP and I was in business. Even implemented this at work for our SMTP server, cut the spam in half!

Add to that some more links, a news page, a links page and a little bit of promotion here and there and that's DaveProxy as of now.

How I work

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Sun, 21 May 2006 13:28:35 nil

Here's a quick write up about how I work, inspired by the people at LifeHacker who have been putting up similar posts. What desktop software do you use every day?

Every morning I launch three programs, Mozilla Firefox, Mozilla Thunderbird and an RXVT terminal through cygwin. My editor of choice is Vim.

What web sites do you use every day?

I like to keep with my favourite feeds using Bloglines and I'd be lost without Google, Sitepoint Forums and PHP.net. What PDA/personal organizer/system do you use to keep organized?

I aren't very organised and I don't own a PDA. I do have a windows smartphone, but use it for calling people, stuff like that. I've recently been trying out the text file todo list techniques I found on Lifehacker a couple of weeks ago, working alright.

Setting up an adserver

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Sat, 13 May 2006 05:15:19 nil

I recently set up an adserver, purely so I could split an ad spot on the proxy site between two campaigns, to see which one performs better.

Installing phpAdsNew, an open-source ad-server, was very easy and their documentation is excellent.

Creating the publishing zone was easy enough, then I had to create the two campaigns. The two ad networks I wanted to compare, were RightMedia and BannerConnect, which both use the yieldmanager software developed by RightMedia. I set these two up as Advertisers, then created a campaign for each. Then under the campaign you create a banner, which I created as HTML banners. Originally I used the standard yieldmanager tags, but this didn't seem to work properly, I don't think Internet Explorer was rendering the ads. Naturally I didn't notice at first, I don't do Internet Explorer. After changing to the Raw iFrame tags, I started getting plenty of impressions again and it worked out quite well.

I'm planning to move a few other ad zones over to the adserver, it makes sense and should help my acquire more money! Next step is to get the geo-targeting going, I've got nothing to geo-target right now, but should come in useful at some stage.

New website - Keldy's Heroes

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Tue, 18 Apr 2006 11:46:42 nil

I'm planning on putting a full website up for the touring hockey team I play for, Keldy's Heroes. I've just got back from the Blackpool Hockey Festival and got a complaint or two about the lack of the website I promised after last years tour.

It's not complete at all, in fact it's only running some photo album software, zenphoto, which was reccomended by a colleague. It seems small and quick, just what I needed really. DNS should be up by the time anybody reads this and hopefully last years photo's will soon be available.

I plan to add a little bit of team history, news and of course a forum, mainly for players past and present to keep in touch.

http://www.keldysheroes.co.uk

ZCE Diploma Arrives

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Mon, 03 Apr 2006 14:44:12 nil

Finally received my diploma from Zend today. I passed the exam way back in November last year and had emailed Zend twice asking politely, when I would be receiving it, but never heard back from them.

It appears to have taken the best part of nine weeks to get here, for some reason they've marked it 'Surface Transportation Only.'

The diploma came in a little frame and I got a couple of stickers to go with it. Apparently I can claim some free ZCE business cards from some online company aswell.

Vim, CTAGS and PHP 5

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Mon, 13 Mar 2006 04:43:20 nil

I've just discovered CTAGS, thanks to theTop 10 things Vi user need to know about Vim list.

Ctags generates an index (or tag) file of language objects found in source files that allows these items to be quickly and easily located by a text editor or other utility. A tag signifies a language object for which an index entry is available (or, alternatively, the index entry created for that object).

To use CTAGS with PHP 5, I downloaded the CTAGS source code, and this patch. Bascially the patch adds capabilities for the private, protected and public visibilty keywords.

# wget http://kent.dl.sourceforge.net/sourceforge\ /ctags/ctags-5.5.4.tar.gz



tar -zxvf ctags-5.5.4.tar.gz

wget http://svn.bitflux.ch/repos/public/misc/ctags-php5.patch

mv ctags-php5.patch ctags-5.5.4

cd ctags-5.5.4

Now patch the php.c file

# patch php.c ctags-php5.patch

Configure build and install CTAGS

# ./configure



make

make install

CTAGS is now installed, we can start to use it. I'd recommend creating tag file for different projects seperately, rather than one big one in your home folder or something. So move to your folder of choice and create the tags file.

# cd /path/to/your/project



ctags -R

This may take a while depending on how many files you have. Once it is done, you're ready to start using CTAGS with Vim. The '-t' option for Vim opens the file containing that tag. So..

# vim -t 'MyClass'

... would open the MyClass.php class definition file. Once inside a file, hitting ctrl-] while the cursor is over a function/class name, Vim will attempt to open the file with that tag. Pressing ctrl-t will take you back a step.

MySQL 5 Certification

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Wed, 08 Mar 2006 13:04:04 nil

Over the last few days I've had to send a couple of emails to customer support at Pearson Vue, basically because I couldn't find a way of booking any exam except the one I have already passed, the ZCE. Eventually, once I tried using the site in IE, I got a javascript alert telling me I needed a seperate web account to register for MySQL exams. Unfortunately, when I tried to create a new web account, I was told that they already had a web account matching my details and would I like them to reset my password!

Anyways, they responded to my second email quite quickly and sure enough I logged in and booked the two MySQL Developer Certification exams, the first on 10th April and the second 8th May. Hopefully this will put me under a bit of pressure and make me pick up the Certification Guide once again.

JBoss and Gentoo

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Thu, 02 Mar 2006 04:55:05 nil

After learning and using Java though my 3 years of university, I haven't touched the stuff since. Not because I don't like it, just because I haven't needed to. After a meeting at work the other day I decided to pull my finger out and re-educate myself, particularly towards the enterprise stuff, which they didn't teach me at Uni. If they had I'd probably be loaded right now.

Finding a tutorial, I sat down at my desktop machine and set about creating my first J2EE application. I'm a big fan of Gentoo Linux and try to use it's portage package management as much as possible, making it easy to maintain my system, so rather than download JBoss like the tutorial says, I simply.

# emerge jboss

This went fine, as did firing up jboss and going to the console through my browser. Then I stumbled. Trying to invoke 'startDatabaseManager' wouldn't seem to work. It said it did, but the applet never appeared. An hour later, after customising the deploy XML file for HyperSonic, I was closer to having the database the way it needs to be, but still couldn't get the applet to fire up. I googled for a while and found a handful of people having the same problems, but never for the same reason. In the end I gave up.

The thing is, right now I'm interested in the J2EE side of things, not so much the database, so it's kind of annoying getting stuck so early. I'll spend some time tonight trying to access the HSQLDB another way, if not I'll try and adapt the tutorial to use MySQL!

Total Web Server requests - AWK one liner

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Thu, 23 Feb 2006 01:18:20 nil

I've just started getting in to AWK, due to the massive traffic load on DaveProxy. The one below is set to run on an Combined log format, but I use it on the Squid logs, which act as a HTTPD accelerator.

Total requests: 429972

Not bad in just under 24 hours.

Here it is anyway, should read your access log, print individual IP addresses along with the number of requests and a grand total at the bottom. The first part I read from a Linux.com article (I think) and I added the sum at the end. I am pretty new to this so any suggestions for improvement or variations would be appreciated.

cat access.log | awk '{print $1}' | sort | uniq -c | sort -n | awk 'sum = sum + $1;END {print "\nTotal request s: ",sum}'

My first XML + XSLT

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Tue, 21 Feb 2006 03:27:11 nil

Been playing around a little at work with these. I was given a rather mundane task of presenting a list of constants, pulled out of the sourcecode for our current project on the fly for use by the developers. The php code to do so and generate the XML took about twenty minutes and then I set about investigating XSLT.

Using an XSLT Tutorial and the source code from Google Sitemaps Stylesheets, I knocked up a couple of Stylesheets, including one with sortable table columns. I couldn't get the alternate stylesheet thing to work with firefox though, don't know if I was doing anything wrong.

Would post some code but I might get in trouble.

99 Bottles of Beer - PHP 5

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Mon, 20 Feb 2006 05:28:58 nil

Got an email today from 99 bottles of beer.net regarding the acceptance of my entry!

I wrote it way back in September, I think because the other PHP5 examples didn't show much more than what could've been done in PHP4.x and because I was bored.

Basically there website is collection of programs that produce the lyrics to the '99 bottles of beer' song. Apparently there are currently over 904 variations. This one has to be my favourite of the one's I've seen, despite being generated by a script.

Google Adsense Account Disabled

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Mon, 13 Feb 2006 11:58:25 nil

Apparently because of 'Invalid clicks' my google adsense account has been banned. I'm a little dissapointed to say the least, as far as I know I've done nothing wrong. Here's a copy of the email they sent me.

Hello David Marshall, It has come to our attention that invalid clicks have been generated on the Google ads on your site(s). We have therefore disabled your Google AdSense account. Please understand that this step was taken in an effort to protect the interest of the AdWords advertisers. A publisher's site may not have invalid clicks on any ad(s), including but not limited to clicks generated by: - a publisher on his own web pages - a publisher encouraging others to click on his ads - automated clicking programs or any other deceptive software - a publisher altering any portion of the ad code or changing the layout, behavior, targeting, or delivery of ads for any reason Practices such as these are in violation of the Google AdSense Terms and Conditions and programme polices, which can be viewed at: https://www.google.com/adsense/localized-terms?hl=en_GB https://www.google.com/adsense/policies?hl=en_GB Publishers disabled for invalid click activity are not allowed further participation in AdSense and do not receive any further payment. The earnings on your account will be properly returned to the affected advertisers. Sincerely, The Google AdSense Team

Needless to say I have appealed against this, things had just started to get going and I was earning enough to pay for my hosting. The chances of getting it revoked are slim at best, but it's got to be worth a try.

UK PHP Conference

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Mon, 30 Jan 2006 07:57:40 nil

There's a PHP conference going down in London next week, I was hoping to attend for a couple of reasons, firstly because I've never been to a conference and secondly because they've got some pretty renowned speakers showing up. Unfortunately I've fractured my thumb and have an appointment at the fracture clinic that day, not to mention I can't drive. I'd reccomend anyone interested get themselves a ticket and check it out, doesn't happen all that often.

PHP Conference UK

DaveProxy 2.0

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Mon, 30 Jan 2006 07:43:54 nil

Just spent an hour or so tweaking the server running DaveProxy, mainly the introduction of mod_perl and a couple of little tweaks to Apache. Both together it should help keep it up during the worst periods of traffic.

I decided I'd take this course of action rather than moving over to PHProxy, although CGIProxy is very heavy on resources, using mod_perl should cut out the perl interpreter instantiation for every request and also allows easier monitoring using Monit. While PHProxy is better on the resources side of things, CGIProxy handles more features.

Depending on how well it goes over the next week, I may introduce squid as a cache between Apache and clients.

New theme

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Sun, 22 Jan 2006 08:30:05 nil

So this is the wordpress theme I've been working on. It looks alot worse on a page full of posts, maybe I should fill up my dummy wordpress install. I definitely need some more separation between posts. I'm gonna leave it on here for now as it is, will offer the code once I'm happy with it.

Book Review: Essential PHP Security

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Sun, 22 Jan 2006 07:55:30 nil

I read this book some time ago and as well as re-emphasizing some of the things I already knew about PHP security, it certainly brought a few finer points to my attention.

While the book is quite small at little over a hundred pages, what you get is pure. There's no sidetracking onto this and that, this book is targeted one hundred percent at PHP development.

I'm no book expert, but I would definitely recommend this book to aspiring PHP developers and think it would also benefit some of the more experienced folk out there.

DaveProxy

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Sun, 04 Dec 2005 23:53:47 nil

DaveProxy

DaveProxy, my new online proxy site is up and running. It uses CGIProxy, a free Perl proxy script. The design took me not long and I used GUG Tutorials for the graphics. It'll stay up while I've got the bandwidth available, unless it starts making me loads of money and then I'll keep it up.

Bug in google?

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Wed, 30 Nov 2005 02:40:47 nil

I think I just found what might be a bug in google.

I was trying to search for the correct way to make a Post request to a webserver by searching for 'http post'. I accidently typed 'http postt'. The usually very helpful 'Did you mean' link was broken. Naturally I pressed back and clicked it again, being too lazy to go back to the search bar and again it was broken. Looking at the URL, something didn't look right.

I tried a couple more searches with spelling mistakes and it seems that if the first word is 'http', the url doesn't get formed properly. See for yourself below.

Google Search for 'http postt'

Zend Certified Engineer

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Mon, 28 Nov 2005 08:26:20 nil

During my well earned week off work, I studied for, took and passed the Zend Certified Engineer Exam.

I was quite suprised when the words 'Congratulations...' appeared on the screen right after I pressed the 'End Exam' button, not because I thought I'd failed but because I was expecting to have to wait for the results. One guy had written that he had to wait two weeks to find out.

The exam went fine really and I felt reasonably confident.

I'd definitely recommend it to any reasonably experienced PHP developers out there. I've only been doing PHP about 18 months and only the last 12 or so full time.

One thing I would say about the exam, would it harm to have syntax highlighting on the example code? After coding for so long in environments like Vim and jEdit, I find it difficult to actually read code. Obviously highlighting errors would be out of the question. Another thing might be the use of better variable names within the example code, maybe using real world examples rather then $my_value.

Passing the exam has spurred me on to try the MySQL Certification ordered the books from Amazon and they arrived today. I also bought Chris Shiflett's Essential PHP Security, once I've had a look at them both I'll post thoughts and reviews.

Certificate Authentication

Welcome to DaveDevelopment

dave.marshall@atstsolutions.co.uk (Dave Marshall) — Mon, 28 Nov 2005 08:05:42 nil

DaveDevelopment is going a place for me to log anything and everything to do with the software and web development work I do, along with other IT related items I come across on the internet.

Maybe I'll teach someone something, maybe others will point out all the mistakes I'm making.

It's not looking upto much now, I'll probably adapt the theme I made for DaveDVD and get it up on here at somepoint and I'll try to write some about pages too.