Squiz Labs Blog - The latest news from the R&D division of Squiz®

PHP_CodeSniffer development moved to Git

Quite a lot of PEAR packages are being moved to hosting on Github, and Squiz Labs is starting to host projects over there as well, so I've moved development of PHP_CodeSniffer to Github and will be closing down the existing SVN repository.

The major benefit of this change is easier patch submission and acceptance. A lot of great developers submit features and bug fixes for PHP_CodeSniffer and I'm sure many will like the ability to submit pull requests instead of packaging up diff files.

Bug tracking remains on the PEAR website for now, and I'll of course continue accepting diffs on issue reports as I currently do.

You'll find PHP_CodeSniffer here: https://github.com/squizlabs/PHP_CodeSniffer

The README has all the info you need to get PHP_CodeSniffer installed from either PEAR or Git, as well as links to the bug tracker and news sources.

PHP_CodeSniffer 1.3.1 released

PHP_CodeSniffer version 1.3.1 has just been uploaded to PEAR and is now available to install. This release has been focused primarily on bug fixing, but given the amount of time since the 1.3.0 release, there were also a few features that made it in. Some of the highlights in this release are:

• A new blame report for Mercurial (--report=hgblame)
• Removed all dependencies on PEAR so PHP_CodeSniffer can run completely from an SVN checkout
• The ability to check code from STDIN
• Relative path support for command line report arguments
• 19 bug fixes

Special thanks to the 10 developers who directly contributed code to this branch. Your help is always very much appreciated.

You can view the full changelog on the PHP_CodeSniffer download page.

Stay up to date on all PHP_CodeSniffer changes, including new features and releases, by subscribing to the RSS feed or following me on Twitter.

PHP_CodeSniffer 1.3.0 released

After more than 12 months of development, it's finally done. PHP_CodeSniffer version 1.3.0 (stable) has just been uploaded to PEAR and is now available to install.

Only bug fixes have been added since the 1.3.0RC2 release, but the changes between the 1.2 and 1.3 branches are numerous. If you're just catching up with this new branch, read the following release posts (and the articles linked within) to get an idea of what has changed:

• PHP_CodeSniffer 1.3.0a1 released
• PHP_CodeSniffer 1.3.0RC1 released
• PHP_CodeSniffer 1.3.0RC2 released

Thanks to everyone who has downloaded and tested the 3 previous releases on this branch. Despite not being stable, these test releases were downloaded more than 8000 times, providing invaluable testing time for such a large architectural change.

And a special thanks to the 13 developers who directly contributed code to this branch. Your help is always very much appreciated.

Developers who have built custom coding standards are reminded to read the 1.3.0 upgrade guide to ensure your standards work with this new version.

You can view the full changelog on the PHP_CodeSniffer download page.

Stay up to date on all PHP_CodeSniffer changes, including new features and releases, by subscribing to the RSS feed or following me on Twitter.

Closure support in PHP_CodeSniffer

Quite a few sniffs included with PHP_CodeSniffer check various aspects of function declarations, from commenting to spacing and everything in between. But the rules for anonymous functions (closures) tend to be different. They are used inside function calls or assigned to variables mid-function. It doesn't make sense to enforce the same commenting and spacing rules in these contexts. Doing so would probably make the code harder to read, which is not the goal of a coding standard.

PHP's tokenizer does not have a token specifically for closures, so for this code:

<?php
function myFunc($foo)
{
    $callback = function ($bar) use ($foo) {
        $bar += $foo;
    };

}//end myFunc()
?>

You end up with a token stack like this:

 *** START SCOPE MAP ***
Start scope map at 1: T_FUNCTION => function
Process token 2 []: T_WHITESPACE =>  
Process token 3 []: T_STRING => myFunc
Process token 4 []: T_OPEN_PARENTHESIS => (
* skipping parenthesis *
Process token 7 []: T_WHITESPACE => \n
Process token 8 []: T_OPEN_CURLY_BRACKET => {
=> Found scope opener for 1 (T_FUNCTION)
...
Process token 11 [opener:8;]: T_VARIABLE => $callback
Process token 12 [opener:8;]: T_WHITESPACE =>  
Process token 13 [opener:8;]: T_EQUAL => =
Process token 14 [opener:8;]: T_WHITESPACE =>  
Process token 15 [opener:8;]: T_FUNCTION => function
* token is an opening condition *
* searching for opener *
        ...
        Process token 27 []: T_OPEN_CURLY_BRACKET => {
        => Found scope opener for 15 (T_FUNCTION)
        ...
        Process token 38 [opener:27;]: T_CLOSE_CURLY_BRACKET => }
        => Found scope closer for 15 (T_FUNCTION)
...
Process token 42 [opener:8;]: T_CLOSE_CURLY_BRACKET => }
=> Found scope closer for 1 (T_FUNCTION)
*** END SCOPE MAP ***

Notice that there are two T_FUNCTION tokens in there (tokens 1 and 15). The second is actually a closure, but sniffs can't tell the difference between them.

Instead of forcing sniff developers to check each function to see if it has a name, PHP_CodeSniffer, from version 1.3.0RC3 (not yet released) will have a new token to listen for: T_CLOSURE.

After the main token processing, the PHP tokenizer built into PHP_CodeSniffer does some additional work. You'll now see this at the end of PHP_CodeSniffer's verbose output:

*** START ADDITIONAL PHP PROCESSING ***
* token 15 on line 4 changed from T_FUNCTION to T_CLOSURE
*** END ADDITIONAL PHP PROCESSING ***

This means that all existing sniffs that listen for the T_FUNCTION token will now ignore closures by default. For all sniffs included with PHP_CodeSniffer, this is the correct behaviour. But if you have been making assumptions in your sniffs that closures also appear as T_FUNCTION tokens, you'll need to make a change to support 1.3.0. Just make sure you register both T_FUNCTION and T_CLOSURE tokens in your sniff, or when searching for next and previous tokens, make sure you pass an array with T_FUNCTION and T_CLOSURE instead of a single token.

This change also means you can start writing targeted sniffs to enforce rules for closures. Coding standards tend to be fairly relaxed on how they can be used and how they should be defined, leading to a variety of different styles. If you are already enforcing a specific style in your standard, and you are using closures, it might be a good time to sit down and think about what standards you want to put around them.

If you can't wait for the next release, you can play around with this new feature by installing PHP_CodeSniffer from the SVN source. Note that this will replace your existing PHP_CodeSniffer install:

pear uninstall PHP_CodeSniffer
svn co http://svn.php.net/repository/pear/packages/PHP_CodeSniffer/trunk PHP_CodeSniffer
cd PHP_CodeSniffer
pear install -f package.xml

Stay up to date on all PHP_CodeSniffer changes, including new features and releases, by subscribing to the RSS feed or following me on Twitter.

PHP_CodeSniffer 1.3.0RC2 released

PHP_CodeSniffer version 1.3.0 release candidate 2 has just been uploaded to PEAR and is now available to install. This release has been focused primarily on bug fixing, but (typically) there were also a few features that made it in. Some of the highlights in this release are:

• Support for multiple output renderers
• The ability to show progress neatly during a run
• The ability to set php.ini values from the command line
• Rule-based exclude patterns
• Google ClosureLinter support
• 16 bug fixes

I'd like to especially thank the 6 generous patch contributors for this release and all those who took the time to submit bugs and test the fixes.

All new features have been documented and will be available on the PHP_CodeSniffer documentation page once the documentation regenerates over the weekend.

Developers who have built custom coding standards are reminded to read the 1.3.0 upgrade guide to ensure your standards are ready for the 1.3.0 stable release. No release date is set, but it's a fairly easy upgrade process and doing it early will give you plenty of time for testing.

You can view the full changelog on the PHP_CodeSniffer download page.

Stay up to date on all PHP_CodeSniffer changes, including new features and releases, by subscribing to the RSS feed or following me on Twitter.

Multiple output renderers in PHP_CodeSniffer

One of the oldest PHP_CodeSniffer feature requests, added back in 2007, is one to allow PHP_CodeSniffer to produce multiple reports after a run. PHP_CodeSniffer provides a number of reporting options, including writing the report to a file as well as the screen, but you've only been able to select one report type to view.

I'm happy to report that this feature has finally been completed, is now available in SVN and will be available in the next release.

How does it work?

Using the new feature is pretty easy. Previously, you'd specify a report type like this:

phpcs --report=summary

This still works, but you can also specify reports like this:

phpcs --report-summary --report-source

This tells PHP_CodeSniffer to print two reports to the screen.

You can also print reports to a file:

phpcs --report-summary=/tmp/summary.txt --report-source=/tmp/source.txt

Or even mix and match:

phpcs --report-summary --report-source=/tmp/source.txt

Note that if you are printing reports to the screen, they will be printed in the order that you specify them on the command line. So if you'd like to see a full report with a summary at the bottom, you would use the command:

phpcs --report-full --report-summary

Why is this useful?

It's all about saving time. If you need to view more than one report, you're now saving yourself time with this feature.

If you are checking a lot of files, PHP_CodeSniffer might take a while to run and might produce quite a long error report. Sometimes you just want a summary of the files that have errors so you can assign files to different developers. Sometimes you want the complete list of errors to act as a todo list. Often, you want both. So now, you can print both the full and summary reports in one run rather than having to process your files twice.

Build systems also benefit from this feature. A build system will often capture the output produced by PHP_CodeSniffer and display it somewhere in the interface, but it will also source the full error list from a file, often in an XML or CheckStyle format. Previously, you'd have the same XML-based report displayed in the output as you did in the file, but now you can have your build system show a summary of the errors from the output and still print the complete list to a file for post-processing.

How do I get it?

If you can't wait for the next release, you can play around with this new feature by installing PHP_CodeSniffer from the SVN source. Note that this will replace your existing PHP_CodeSniffer install:

pear uninstall PHP_CodeSniffer
svn co http://svn.php.net/repository/pear/packages/PHP_CodeSniffer/trunk PHP_CodeSniffer
cd PHP_CodeSniffer
pear install -f package.xml

Rule-based exclude patterns

PHP_CodeSniffer has been able to exclude files and directories while checking large code bases for quite some time, but it has always been an "all or nothing" feature. If the file is excluded then it will not generate any errors or warnings.

A feature to exclude specific sniffs for specific files has been requested a couple of times, but the suggested implementations have always been around the idea of adding comments into your source code. I'm not a big fan of annotating your source code for external tools, so I've never really looked into this feature.

But now that PHP_CodeSniffer uses ruleset.xml files, a recent feature request prompted me to take another look. A it turns out, the ruleset format is the perfect for defining these sort of custom rules.

I've just committed this feature into the PHP_CodeSniffer SVN repository. The documentation wont be updated until the weekend, so here is some information about how to add it to your ruleset.xml file now:

<!--
   You can also hard-code ignore patterns for specific sniffs,
   a feature not available on the command line.

   The code here will hide all messages from the Squiz DoubleQuoteUsage
   sniff for files that match either of the two exclude patterns.
-->
<rule ref="Squiz.Strings.DoubleQuoteUsage">
   <exclude-pattern>*/tests/*</exclude-pattern>
   <exclude-pattern>*/data/*</exclude-pattern>
</rule>

<!--
   You can also be more specific and just exclude some messages.

   The code here will just hide the ContainsVar error generated by the
   Squiz DoubleQuoteUsage sniff for files that match either of the two
   exclude patterns.
-->
<rule ref="Squiz.Strings.DoubleQuoteUsage.ContainsVar">
   <exclude-pattern>*/tests/*</exclude-pattern>
   <exclude-pattern>*/data/*</exclude-pattern>
</rule> 

Closure Linter Support in PHP_CodeSniffer

Last week, Google announced that they had open sourced a new tool, Closure Linter, to enforce their JavaScript style guide. Closure Linter checks for a range of common JS and Google style errors and includes some checking in there for JSDoc comments as well.

The timing couldn't have been much better as we were just discussing the need for new Squiz sniffs to enforce JsDoc comments for our JavaScript files. Re-writing the comment parser in PHP_CodeSniffer to support the different tags and syntax that JsDoc uses was not something easily achievable. So instead, we have added a new sniff that runs Closure Linter on each JS file and reports errors and warnings through PHP_CodeSniffer.

The output looks a bit like this:

----------------------------------------------------------------------
FOUND 4 ERROR(S) AND 4 WARNING(S) AFFECTING 6 LINE(S)
----------------------------------------------------------------------
  17 | WARNING | gjslint says: (0200) Invalid JsDoc tag: package
  18 | WARNING | gjslint says: (0200) Invalid JsDoc tag: subpackage
  20 | WARNING | gjslint says: (0200) Invalid JsDoc tag: copyright
  22 | WARNING | gjslint says: (0225) Missing @this JsDoc in function
     |         | referencing "this". (this usually means you are
     |         | trying to reference "this" in a static function, or
     |         | you have forgotten to mark a constructor with
     |         | @constructor)
  24 | ERROR   | Missing docs for parameter: "id"
  24 | ERROR   | Missing docs for parameter: "screenid"
  24 | ERROR   | Missing docs for parameter: "system"
 114 | ERROR   | Additional whitespace found at end of file
----------------------------------------------------------------------

By default, all errors produced by Closure Linter are added as warnings and start with gjslint says so you can easily ignore these if they are incorrect. You'll also notice that the warning messages include the error code in brackets, as well as the message. This error code is the internal code used by Closure Linter and has nothing to do with PHP_CodeSniffer itself, although you can use it to customise your own standard. You can view a full list of the error codes, along with a very basic description of each, in the errors.py source file.

This new generic sniff was designed to allow for flexible customisation. You can specify a list of Closure Linter error codes to both completely ignore and report as errors instead of warnings. For the Squiz coding standard, we ignore some warnings that conflict with our own standards and have some JsDoc-based warnings reported as errors. We also change the format of those important messages to remove the gjslint says prefix so they fit in more easily with the other errors being reported.

All this is done easily using the new ruleset.xml format in PHP_CodeSniffer 1.3.0. This snippet comes from the ruleset.xml file for the Squiz standard and has been committed to the PHP_CodeSniffer SVN repository, along with the new ClosureLinter sniff to do the work.

<!-- We don't want gsjlint throwing errors for things we already check -->
<rule ref="Generic.Debug.ClosureLinter">
 <properties>
  <property name="errorCodes" type="array" value="0210"/>
  <property name="ignoreCodes" type="array" value="0001,0110,0240"/>
 </properties>
</rule>
<rule ref="Generic.Debug.ClosureLinter.ExternalToolError">
 <message>%2$s</message>
</rule>

Installing Closure Linter is pretty easy (install guide here). Once installed, you'll have a new gjslint command available to you. To tell PHP_CodeSniffer where gjslint is installed, run the following command:

phpcs --config-set gjslint_path /path/to/gjslint

Note that you will need to install PHP_CodeSniffer from the latest SVN checkout for this to work and the Squiz standard is the only current standard to use this new sniff. To check a JavaScript file using the Squiz standard, run the following command:

phpcs --standard=Squiz /path/to/file.js

This new sniff is another great reason to plan your migration to PHP_CodeSniffer 1.3.0 (currently in beta) including the conversion of any custom standards to the new ruleset.xml format. Please read the custom coding standard upgrade guide for more information.

PHP_CodeSniffer 1.3.0RC1 released

PHP_CodeSniffer version 1.3.0 release candidate 1 has just been uploaded to PEAR and is now available to install. This release has been focused on bug fixing and general stabilising of the architectural changes released in 1.3.0a1, but there were also a few nice features that made it in.  Some of the highlights in this release are:

• Support for exclude-pattern tags in ruleset.xml files
  
• A new --encoding command line argument to fix UTF-8 encoding issues in some reports
  
• All reported bugs are now fixed

New features have been documented and will be available on the PHP_CodeSniffer documentation page once the documentation regenerates over the weekend. This includes an updated coding standard tutorial and an annotated ruleset.xml file to take you through all the features of this new format and how to use them.

Developers who have built custom coding standards are encouraged to read the 1.3.0 upgrade guide to ensure your standards are ready for the 1.3.0 stable release. No release date is set, but it's a fairly easy upgrade process and doing it early will give you plenty of time for testing.

You can view the full changelog on the PHP_CodeSniffer download page.

Upgrading custom coding standards to support version 1.3.0

PHP_CodeSniffer version 1.3.0 is currently in alpha, but it contains an important backwards compatibility break that all coding standard authors need to be aware of. Upgrading your coding standard to make use of the new ruleset.xml files is an easy process and this guide will show you how to get it done.

Please note that if you have not created your own coding standard, you do not need to follow this guide. Users of PHP_CodeSniffer that use one of the built-in standards can continue to check their code as normal.

This guide assumes your coding standard has the following directory structure, which is based off the coding standard tutorial in the PHP_CodeSniffer documentation:

MyStandard
|_ MyStandardCodingStandard.php
|_ Sniffs
   |_ Commenting
      |_ DisallowHashCommentsSniff.php

In this sample coding standard, we have a single sniff being included directly within the standard's sniff directory. But it doesn't matter how many sniffs you have as they are automatically included into the standard in all versions of PHP_CodeSniffer. Your standard may not even have any directly included sniffs, preferring to include sniffs from other standards via the CodingStandard.php class.

So the only thing we need to do to upgrade a custom coding standard is convert the CodingStandard.php class to a ruleset.xml file. We can leave all directly included sniffs alone and we don't have to change our directory structure.

The Basics

The first thing you need to do is create a ruleset.xml file directly under your top-level directory. The name of the file must be ruleset.xml

touch MyStandard/ruleset.xml

The contents of this file will be minimal if you are not including any sniffs from other standards. So the file content would look like this:

<?xml version="1.0"?>
<ruleset name="My Standard">
 <description>My custom coding standard</description>
</ruleset>

A simple ruleset.xml file like this tells PHP_CodeSniffer that this directory contains a coding standard, the name of the standard is My Standard and the sniffs in the standard are sourced directly from the default Sniffs directory.

Once you've created your ruleset.xml file, you can go ahead and delete the CodingStandard.php class file as it is no longer required. However, you can keep both files in the coding standard if you want to use your standard in both old and new versions of PHP_CodeSniffer. But be aware that you will need to make changes to both files and any advanced ruleset features you add to your ruleset.xml file can not be replicated in your CodingStandard.php class file.

A Simple Coding Standard Class

You can build on a coding standard by including sniffs from other standards. A sample CodingStandard.php class from the PHP_CodeSniffer documentation contains this:

public function getIncludedSniffs()
{
  return array(
          'PEAR',
          'Generic/Sniffs/Formatting/MultipleStatementAlignmentSniff.php',
          'Generic/Sniffs/Functions',
         );

}//end getIncludedSniffs()

The example above includes the MultipleStatementAlignment sniff from the Generic coding standard, all sniffs in the Functions category of the Generic coding standard and all sniffs defined in the PEAR coding standard.

These rules would be replicated in a ruleset.xml file like this:

<?xml version="1.0"?>
 <ruleset name="My Standard">
 <description>My custom coding standard</description>
 <rule ref="PEAR"/>
 <rule ref="Generic.Formatting.MultipleStatementAlignment"/>
 <rule ref="Generic/Sniffs/Functions"/>
</ruleset>

The two changes here are the use of the rule tag to include sniffs and standards and also the way we reference an individual sniff. Instead of specifying the path to the sniff we instead specify the internal code that PHP_CodeSniffer gives it, which is based on the path. It's actually a pretty easy conversion. Just just drop the Sniffs directory, convert the slashes to periods and remove Sniff.php from the end. Here are some more examples to make sure it is clear.

BEFORE: Generic/Sniffs/VersionControl/SubversionPropertiesSniff.php
AFTER:  Generic.VersionControl.SubversionProperties

BEFORE: PEAR/Sniffs/ControlStructures/ControlSignatureSniff.php
AFTER:  PEAR.ControlStructures.ControlSignature

BEFORE: Squiz/Sniffs/Strings/DoubleQuoteUsageSniff.php
AFTER:  Squiz.Strings.DoubleQuoteUsage

Coding Standards with Exclusions

Some coding standards include a set of sniffs from an external standard but then exclude some specific sniffs. A sample CodingStandard.php class with exclusions from the PHP_CodeSniffer documentation contains this:

public function getIncludedSniffs()
{
  return array(
          'PEAR',
         );

}//end getIncludedSniffs()

public function getExcludedSniffs()
{
  return array(
          'PEAR/Sniffs/ControlStructures/ControlSignatureSniff.php',
         );

}//end getExcludedSniffs()

The example above includes the whole PEAR coding standard except for the ControlSignature sniff.

These rules would be replicated in a ruleset.xml file like this:

<?xml version="1.0"?>
<ruleset name="My Standard">
 <description>My custom coding standard</description>
 <rule ref="PEAR">
  <exclude name="PEAR.ControlStructures.ControlSignature"/>
 </rule>
</ruleset>

Notice how the exclusions are grouped with the standard they are being excluded from and how they again use the internal sniff codes instead of full paths to sniff files.

A Practical Example

PHP_CodeSniffer comes with a coding standard called PHPCS, which is the entire PEAR standard as well as some Squiz sniffs thrown in. In version 1.2.2, the PHPCSCodingStandard.php file contains these two functions:

public function getIncludedSniffs()
{
  return array(
          'PEAR',
          'Squiz',
         );

}//end getIncludedSniffs()

public function getExcludedSniffs()
{
  return array(
          'Squiz/Sniffs/Classes/ClassFileNameSniff.php',
          'Squiz/Sniffs/Classes/ValidClassNameSniff.php',
          'Squiz/Sniffs/Commenting/ClassCommentSniff.php',
          'Squiz/Sniffs/Commenting/FileCommentSniff.php',
          'Squiz/Sniffs/Commenting/FunctionCommentSniff.php',
          'Squiz/Sniffs/Commenting/VariableCommentSniff.php',
          'Squiz/Sniffs/ControlStructures/SwitchDeclarationSniff.php',
          'Squiz/Sniffs/Files/FileExtensionSniff.php',
          'Squiz/Sniffs/NamingConventions/ConstantCaseSniff.php',
          'Squiz/Sniffs/WhiteSpace/ScopeIndentSniff.php',
         );

}//end getExcludedSniffs() 

To support version 1.3.0, the ruleset.xml file was created with this content:

<?xml version="1.0"?>
<ruleset name="PHP_CodeSniffer">
 <description>The coding standard for PHP_CodeSniffer.</description>

 <!-- Include the whole PEAR standard -->
 <rule ref="PEAR"/>

 <!-- Include most of the Squiz standard -->
 <rule ref="Squiz">
  <exclude name="Squiz.Classes.ClassFileName"/>
  <exclude name="Squiz.Classes.ValidClassName"/>
  <exclude name="Squiz.Commenting.ClassComment"/>
  <exclude name="Squiz.Commenting.FileComment"/>
  <exclude name="Squiz.Commenting.FunctionComment"/>
  <exclude name="Squiz.Commenting.VariableComment"/>
  <exclude name="Squiz.ControlStructures.SwitchDeclaration"/>
  <exclude name="Squiz.Files.FileExtension"/>
  <exclude name="Squiz.NamingConventions.ConstantCase"/>
  <exclude name="Squiz.WhiteSpace.ScopeIndent"/>
 </rule>
</ruleset>

Questions?

This guide has hopefully provided you without enough information and examples to help you convert your custom coding standards to support PHP_CodeSniffer 1.3.0. If you have any questions, please ask them in the comments section below or contact Greg Sherwood directly via the PEAR website.