We ran into a little problem at work this afternoon. Running a full set of Behat features caused one of the scenarios to fail in the middle – it was a scenario detailing user signup via an API, where the response indicated that the expected security token was from a different user object. Strange as the database indicated there was only a single entity that had been persisted. Running the scenario on its own, everything was fine.

The solution was discovered via this helpful StackOverflow post (yay for SO). It seems that Doctrine 2 keeps a reference to persisted entities, to save time looking them up again. Which we’d explicitly wanted to do. Since our scenarios have a drop-and-reload-style setup, we didn’t want any previous entities hanging around. Based on the StackOverflow answer, the solution was to adjust the end of our setUp() method to add the last line shown in the code example below:

/**
 * Sets up our database
 */
protected function setUp()
{
    // Our usual setup stuff goes here
    // eg drop tables, load schema, any
    // fixture loading and so on.
    $this->dropTables();
    $this->generateSchema();
 
    // Here we now detach any previously-queried entities
    // to avoid caching issues.
    $this->em->clear();
}

where $this->em is an instance of the Doctrine EntityManager. Simples.

At the weekend I was experimenting with ZeroMQ (0MQ for short here) for handling requests for items in an online shop. The idea behind it was simple – schedule requests for items to be added to a shopping cart, and don’t allow overselling of said stock items. The legacy code handles requests by wrapping in a transaction and checking for the amount of items currently available (sales + in existing carts), then adding/rejecting as appropriate.

If however there is significant traffic, this can result in oversells as the number of items available can be reported the same to more than 1 user. If both users request the final item, your oversell happens. I wanted to remove the likelihood of this happening, and so I turned to a queueing system to see how this could make a difference. I’d seen the video of Ian Barber’s presentation at PHP UK last year, and liked the apparent simplicity of 0MQ coupled with the lightweight in-memory approach so decided to experiment a bit.

After downloading and installing the 0MQ library, and the pecl extension for PHP, I was all set to go.

The idea seemed simple:

• Take all requests for items, and pop them into a queue
• If a response from the queue isn’t available immediately, the customer should be shown a page which polls periodically for the result.
• When the result is available, redirect the customer to their cart (success) or back to the item page (failure)

The 0MQ documentation at first seemed thorough, but reading through I realised it was a bit too verbose for my liking. I can appreciate the use of metaphors and friendly language, but it seemed to hinder understanding of what 0MQ could offer, coupled with everything being on the same page. Slightly overwhelming! In contrast, RabbitMQ’s tutorial pages seemed to be a lot clearer. For a developer new to queueing system, this was definitely a win for RabbitMQ – a case of which-image-represents-best-what-I-want *click* etc. However, I stuck with 0MQ for now for the reasons outlined above.

I settled on the Push-Pull methodology, “Divide and Conquer” in the 0MQ docs. This would satisfy the requirements above of being able to dump a request into a queue and periodically (via another mechanism) check for a result. I’ve shamelessly ripped off the accompanying image from their docs here in order to illustrate my point :-)

In the above image, the part labelled “Ventilator” can be equated to the user requesting a quantity of items to add to a shopping cart, and my queue handler is a single instance of a worker. In my implementation, I went for something like the following process:

• User requests 1 of item A via form
• Memcache entry is created with a unique key for the user’s request
• 0MQ request is created, “allocate me 1 of item A, for my session ID X and unique key ABC”
• … polling happens here …
• 0MQ worker receives message and performs relevant checks
• Memcache value for supplied unique key is updated with response
• Polling for this updated value causes a redirect for the user accordingly

Polling checked the memcache key for a successful response. I decided to use memcache for this as it was quick to get going and I only technically needed a key-value setup. I also wanted to avoid unnecessary hits on a database if possible, preferring to keep this for transactional data. Plus I also wanted to play with memcache a bit ;-)

Testing

In tests, this setup seemed to work well, and the updated memcache value ended up being a serialized array with a success/error code, and some additional information eg “Sold out”, “Item is unavailable”, “You requested 2 but only 1 is available”. This enabled better feedback for the user when their request was unsuccessful.

Running this in small-scale tests was great. It was good to be able to start and stop the queue handler, and see the user being automatically bounced to the “please wait” polling page whenever the queue handler wasn’t running or had a backlog of messages to process. Fire up the handler, checks take place and the user is then redirected accordingly.

When running this under heavier loads however, we hit instances where occasionally (due to intentionally-bad Apache settings), the server processes ran out of memory and required a restart of Apache. This was done deliberately to see how the queue could recover from different scenarios. As a result, restarting Apache would cause a flood of messages to be sent to the queue handler in one go, presumably because they’d been blocked somehow previously with the large load. This was confusing and unexpected behaviour to me, however I presumed this was because Apache (with mod_php) had been struggling to send the messages out under the load, and a restart caused a flush of some sorts. I also presume this isn’t down to any fault of 0MQ. It does however pose the question of what to do in a scenario where messages can potentially be lost, but this could be solved by a message queue that persists messages somewhere until completion.

Despite this teething issue which wasn’t really down to any fault of 0MQ, the system works great. Being new to queueing like this, I’m not entirely sure how this would scale and still keep the ability to not oversell an item – database transactions from multiple workers would need to be synchronised somehow to ensure that figures are kept up to date. But overall, I’m very happy with it as a starting point for a replacement solution.

Mini post. I can never find this information on the PEAR site, so this is just a reference for me really :-)

After all code changes have been made and pushed into the repository, do the following

[rich@localhost]# pfm
* Follow instructions here, eg update version number, release notes
* Regenerate Contents
* Save & Exit
[rich@localhost]# git add package.xml
[rich@localhost]# git commit
[rich@localhost]# git tag "RELEASE-x.y.z"
[rich@localhost]# pear package
* Package file is created, release this through PEAR website as normal

I’ve been experimenting with sending emails but rendering the content with Twig. This allows emails to extend a standard layout where required, but also gives us the flexibility to insert other content into the template which may be required. In the first instance, we’re interested in supplying the email subject here.

The concept is simple – within Symfony2, firstly get Twig from the container or inject it accordingly. You can then render the template as a whole, and also extract any other items required. My example is taken from a listener class I’ve been using, but the concepts can be applied appropriately depending on where you want to use it:

class MyListener
{
    protected $mailer;
    protected $twig;
 
    public function __construct(\Swift_Mailer $mailer, \Twig_Environment $twig)
    {
        $this->mailer = $mailer;
        $this->twig = $twig;
    }
 
    public function onMyevent($myEvent)
    {
        $emailTo = $myEvent->getEmailTo();
 
        // Load the template in
        $templateFile = "MyBundle:emails:myEmailTemplate.twig";
        $templateContent = $this->twig->loadTemplate($templateFile);
 
        // Render the whole template including any layouts etc
        $body = $templateContent->render(array("someParam" => "foo"));
 
        // Get the subject block out
        // If the block doesn't exist, use a default
        $subject = ($templateContent->hasBlock("subject")
            ? $templateContent->renderBlock("subject", array("param1" => $someInformation->foo))
            : "Default subject here");
 
        $subject = trim($subject);
 
        // Send email
        $message = \Swift_Message::newInstance()
            ->setSubject($subject)
            ->setFrom("info@example.com")
            ->setTo($emailTo)
            ->setBody($body)
        ;
        $this->mailer->send($message);
    }
}

This means we can put something like the following in our template file:

{% extends "MyBundle:emails:emailLayout.twig" %}
 
{# This block is rendered in emailLayout.twig #}
{% block body %}
Hello, world. This is your email content here
{% endblock %}
 
{% block subject %}
Your subject line goes here
{% endblock %}

I was hoping to use the Twig {% spaceless %} tag, but it appears that it’s only available for HTML (sadly this PR has been closed; shame as it would have been useful). Hence the trim() call when retrieving the subject.

Of course, you can extend the above to inject in default email-FROM-addresses and names, template names and so on where appropriate.

Update: Thanks to Fabien for pointing out (comments below) that you can remove whitespace by using the following (hyphens at the end of the opening block tag and at the start of endblock:

{% block subject -%}
Your subject line goes here
{%- endblock %}

I’ve been thinking about next year’s New Year’s Resolutions again. One which I’m going to repeat from this year was to blog more (inspired by Lorna’s post about blogging more back in January), but also a new one I want to achieve is to speak at a conference.

I’ve presented papers at a couple of non-web related conferences a number of years ago, but nothing in the web field, and I’m unsure as to the correct approach to take with this. My main reason for wanting to speak is that I want to be able to share knowledge I’ve learnt in the hope that this may be of interest to at least one person. I’m also passionate about the technology I use in day-to-day development and would most probably like to include this. My main concern however is “would it be interesting enough?”, which I’m inclined to feel is a catch-22 situation – I won’t know until presenting it, but then I won’t get a chance to present it unless I submit.

Calls for papers are continuously springing up, and I’m wondering about the best approach here to get my foot in the door so to speak. Is it a case of writing up something that I feel is interesting and would benefit others, and just taking the leap? Finding local Uncons/user groups and trying the material out there?

I’d be interested to hear any tips and pointers :-)

NB: Cross-posted on our work blog at White October :-)

I spent some time last week working on automated MailChimp integration with one of our Symfony2 sites. The project uses Behat and Mink extensively for BDD testing, so I wanted to be able to test that when users registered on our site, they were automatically added to the client’s MailChimp distribution list. Unsubscription could happen via a checkbox in the user’s account settings and this needed to work the same way behind the scenes.

I opted to use one of the available MailChimp bundles to aid integration and with a bit of tweaking this worked great. The bundle comes with a stub connection which is able to collate requests sent and make them available for inspection. I wanted to be able to utilise this neatly inside a set of tests, and potentially also make it available in the dev environment. A bit of skimming the Symfony2 cookbook led me to the entry on Symfony2 data collectors.

The concept is simple – a data collector is registered and is able to, well, collect data! For example, the number of queries during a request, or the number of emails sent. This can then be accessed via the profiler in the relevant environment (dev, test etc). In my case I wanted to collect and subsequently inspect all requests sent to MailChimp.

First step – create me a data collector:

< ?php
 
namespace Jirafe\Bundle\MailChimpBundle\DataCollector;
 
use Symfony\Component\HttpKernel\DataCollector\DataCollector;
use Symfony\Component\HttpFoundation\Request;
use Symfony\Component\HttpFoundation\Response;
use Jirafe\Bundle\MailChimpBundle\Connection\ConnectionInterface;
 
class MailChimpDataCollector extends DataCollector
{
    private $connection;
 
    public function __construct(ConnectionInterface $connection)
    {
        $this->connection = $connection;
    }
 
    public function collect(Request $request, Response $response, \Exception $exception = null)
    {
        $this->data = array(
            'requests'  => $this->connection->getRequests(),
        );
    }
 
    public function getRequestCount()
    {
        return count($this->data['requests']);
    }
 
    public function getRequests()
    {
        return $this->data['requests'];
    }
 
    public function getName()
    {
        return 'mail_chimp';
    }
}

The collect() method is the interesting part here – this is called whenever data collection is required – usually once per request. We collect the requests from the connection supplied to the constructor.

Next step, register this during the bundle’s load() method:

// Add connection to data collector
$definition = $container->getDefinition('mail_chimp.data_collector');
$definition->addArgument(new Reference(sprintf('mail_chimp.connection.%s', $config['connection'])));

The above uses the connection type specified in the bundle’s configuration option and previously registered as a service in the same method. If you’re creating your own custom collector, this is just the normal parameters you’d pass in to your service when it’s instantiated.

Finally, I added a method and subsequent implementation to the interface and related connection classes within the bundle, so that MailChimp request objects were collected in an array, and made available to the data collector upon request. This is the getRequests() method listed above in the data collector; called in the collect() method. Quick ‘n’ dirty here, since there would only ever be a single MailChimp request (possibly 2) made in any one request.

Once all this was in place, I could then test correctly. Note that I’d rather have done the other way round (test and then develop) but since this was experimentation, I figured it was OK ;-) In my Behat scenarios, I added the getSymfonyProfiler() method detailed in a Behat/Mink cookbook entry, and proceeded to write my step (Behat 1.x-style):

$steps->Then('/^my email address is unsubscribed from the relevant MailChimp list$/', function($world) {
 
    $profiler = $world->getSymfonyProfiler();
    if (!$profiler)
    {
      throw new \RuntimeException("Symfony2 profiler not present for tests, wrong driver?");
    }
    $mcDataCollector = $profiler->getCollector("mail_chimp");
    $listID = $world->container->getParameter("mailchimp_list_id");
 
    // Check the requests made to make sure we have an unsubscribe
    assertEquals(1, $mcDataCollector->getRequestCount());
 
    $requests = $mcDataCollector->getRequests();
    $req1 = $requests[0];
    assertEquals("listUnsubscribe", $req1->getMethod());
 
    $params = $req1->getParams();
    assertEquals($listID, $params["id"]);
 
    // Fire the redirect on the client
    $world->getSession()->getDriver()->getClient()->followRedirect();
});

The final line here is as a result of disabling redirects in a previous step since the data collector operates on a per-request basis. If you’re subscribing eg via a form POST, then you’ll need to disable redirects prior to submitting the form, otherwise your data collector will be empty when you check it. Disable them and then redirect after you’ve carried out your checks.

The implementation for the above is up on our Github MailChimpBundle fork here; I’m not particularly fond of having to hack about the connection classes, and ideally it would be nice to layer the data collector on top without having to adjust existing code. That said, it does the job fine here, and it’s good to use even more features of Symfony2 ;-)

Came across this one today whilst integrating some new code into my current Symfony2 project. The project has 2 custom Twig extensions – one with some filters in, one with some custom methods. When creating the latter, I’d copied the class from the former and tweaked to suit. In doing so, I’d left these lines in:

    /**
     * Name of this extension
     *
     * @return string
     */
    public function getName()
    {
        return "myfirstextension";
    }

The Twig documentation states that all extensions should be named uniquely, unless you need to replace any existing extensions, in which case you should name them the same. See here for a better explanation :-) Great for extensibility and adjusting extensions that maybe you don’t have control over. Not so good when both extensions do something different and you don’t quite understand why it’s not working….

Anyway, quick question in the #symfony IRC channel and Stof came to the rescue. Give that man a medal :-) and name your extensions uniquely if you don’t want them to overwrite others!

Quick post this, as I know I’ll forget/lose this code otherwise… Sometimes you need the ability to force a logout of a user. The example I’ll use here is a user requesting their account be deactivated – you assume that upon confirmation of the deactivation the user’s model is marked as such, and the user is subsequently logged out and shown a confirmation page:

// Update your model here. Example:
$user = clone $this->get("security.context")->getToken()->getUser();
$user->setEnabled(false);
$user->save();
 
// Log the user out
$this->get("request")->getSession()->invalidate();
$this->get("security.context")->setToken(null);
 
// Flash var to enable the user to see the deactivation success page
$this->get("session")->setFlash('deactivate_account.success', true);
 
// and redirect to success page
return new RedirectResponse($this->get("router")->generate("deactivate_account_success"));

Your deactivation page should allow viewing without requiring authentication (anonymous users can view it). The above code should be self-explanatory – invalidate the current session, and set the security token to null (rather than the previously-authenticated token). The flash variable I set so that viewing of the success page is one-shot rather than being permanently viewable. Just my preference :-)

Just came across this article on ubuntuforums.org which describes how to add a greyscale printing option when using a Canon MP270 printer under Ubuntu. Reprinting here in case the original disappears. Dead simple!:

After reading the manual from Canon, I have made the grayscale printing working too. For those who need it, here is the fast tip:

Under System–>Administration–>Printing

right click MP250 that you have already setup, then select Properties from the popup menu, go to Job Options, scroll down to the bottom, add this option in:

CNGrayscale

click the Add button to add the option to the list.

Beside the CNGrayscale option, type in TRUE in the textfield, then click Apply, and close the Properties dialog.

By doing this, this printer is setup to be Grayscale printing by default. You can definitely have another copy of printer setup, pointing to the same printer, but for color printing. This is what i have done.

I’ve had a case recently whereby I needed to implement a registration form in my current Symfony2 application, based on a Mandango model. This form consisted of selected fields from my model, along with a separate “I accept the terms and conditions” checkbox; pretty standard. In Symfony 1, it was straightforward to add in non-model fields, however in Symfony2 the convention for forms is to create a domain model which represents your form, which then gets populated with the data.

Initially I went with extending from the Mandango model (in my case, Model\MyOwnBundle\User being the original model) and adding in a separate get/setTermsAndConditions() method, but this caused problems when saving the model down, as Mandango couldn’t find the document class (since I’d extended it and called it UserRegistration).

After a bit of digging and experimentation however, the following code does what I need it to, without having to extend my existing model or create a new one (which I’d then have to use to populate my actual Mandango model):

use Symfony\Component\Form\AbstractType,
    Symfony\Component\Form\CallbackValidator,
    Symfony\Component\Form\FormBuilder,
    Symfony\Component\Form\FormError,
    Symfony\Component\Form\FormInterface;
 
class RegisterFormType extends AbstractType
{
    /**
     * Constructs our registration form
     * 
     * @param \Symfony\Component\Form\FormBuilder $builder
     * @param array $options
     * @return void
     */
    public function buildForm(FormBuilder $builder, array $options)
    {
        // Build the form
        $builder->
            add("firstName", "text")->
            add("lastName", "text")->
            add("emailAddress", "email")->
            add("t_and_c",
                "checkbox",
                array(
                    "property_path" => false,
                )
            );
 
        // "True" validator on the form for t&c
        $builder->
            addValidator(new CallbackValidator(function(FormInterface $form)
            {
                if (!$form["t_and_c"]->getData())
                {
                    $form->addError(new FormError('Please accept the terms and conditions in order to register'));
                }
            })
        );
    }
 
    // ...
}

The above adds the t_and_c field but using "property_path" => false means it’s not validated against the model and therefore no need to implement separate get/set() methods. Instead we use a callback validator on the form to check the value of the t_and_c field and ensure it’s been checked. Et voila :-)