Slides for my talk “Scaling Communication via Continuous Integration” are now available on Slide Share.
As presented at PHPUK2012 in London.
New Addition to CSRunner: php -l
Speed is addictive.
After speeding up PHP Code Sniffer runs, the slowest job for Etsy was running php -l on all the files.
So, in short, the latest release of CSRunner, version 0.2.1, will execute lint on the recently changed files.
You can also run the lint command separately, but why would you want to do this instead of your precious find piped to a php -l script?
Because this command outputs to checkstyle format!
Which means if you are Jenkins and using the Jenkins Checkstyle Plugin, you can add the resulting checkstyle xml to your list of Checkstyle results. Voila!
Enjoy!
Thoughts on Accessibility with Regards to Testing
In Alternative Thoughts on Sniffs for Useless Method Overriding I discussed the discovery of a horrible dance that some developers will do just to test private functions:
- See a
privatemethod you want to test in isolation - Upgrade the
privatetoprotected - Create a child class implementation of this class in the
TestCasefile - Override any
protectedmethods, that were actually intended to beprivatemethods and make them nowpublic - Test the new child class instead of the actual class
This violates the only one class per test file rule, as discussed in PHPUnit Coding Standard: Classes
Many claim that they do this because they do not want the methods to be part of the public API, but need to make it protected so they can do this hack. Yet those who argue for this so called solution have nothing to reconcile the fact that they have removed the ability to distinguish between methods that should be override-able by child implementations and those that should not.
In all honesty, if you have functionality that you would prefer not to be in the public API of one object, but it is sufficiently complex to warrant it’s own battery of tests, you should take that as a suggestion to pull that functionality out into it’s own object.
It is a pity to hide something so complex in a private function, and tempt others into exercising copy/paste coding practices to use it.
If you are using PHP 5.3, then you can do another cheat that will let you avoid the number of class violations, and let you keep using the private keyword. Sebastian Bergmann discusses this in Testing Your Privates.
Don’t jump the gun. This is still not license to blindly make things private. Even Sebastian repeats in the post and in the comments:
Just because the testing of protected and private attributes and methods is possible does not mean that this is a “good thing”.
So why am I even talking about these naughty ways to test private methods?
Because I cannot necessarily stop you!
So let me offer an alternative to all of this that should help communicate intent in your code, let you use all the accessibility levels, and keep you from re-hashing reflection calls in your test methods.
Today I put out a new release of the Etsy PHPUnit Extensions. This new release includes PHPUnit_Extensions_Helper_AccessibleObject.
PHPUnit_Extensions_Helper_AccessibleObject is a wrapper for classes which contain private or protected methods that you would like to test in isolation, for some bizarre reason. But, you may only call private and protected methods of the wrapped object that are annotated with @accessibleForTesting.
Why the special @accessibleForTesting annotation on methods in my production code?
With the @accessibleForTesting you will be able to communicate with other developers that while this code is private or protected you intend to test the logic in it. Also, we can easily comb through the code with a static analyzer to find spots in our code that we might want to consider extracting into more reusable places, like another class. Furthermore, we can also sniff for usages of Reflection.* in our tests and useless method overrides, then double back to ensure we move everything to this AccessibleObject method for testing private and protected methods so that we can go back to the loveliness mentioned in the first half of this paragraph.
If you were making the argument for doing the horrible dance to test private methods mentioned in the beginning of this article, then you can certainly get onboard with being more descriptive about your intent with @accessibleForTesting.
For examples, usage, and code: https://github.com/etsy/phpunit-extensions/wiki/Helpers
P.S. This is still naughty! But it’s a compromise I think I might be able to live with.
Mockery Test Case for PHPUnit
Mockery is a wonderful PHP mock object framework. It reminds me of Mockito for Java.
Mockery is far more fluent than MockObjects, the mock object framework packaged with PHPUnit. Mockery and MockObjects can work in the same environment, independent of one another.
Since Mockery is a free standing library, it can be used in PHPUnit, but you need to make sure you either have a loader that supports Mockery or use a bootstrap.php to use the loader packaged with Mockery. You will also need to consider using the TestListener packaged with Mockery to avoid having to remember to call Mockery::close() in tearDown for each TestCase, and a TestListener requires a PHPUnit XML configuration file.
All of this is do-able, but just not fun.
Now there is another alternative:
PHPUnit_Extensions_Mockery_TestCase
Using this new test case eliminates the bootstrap.php, TestListener with XML configuration, and explicit static calls to Mockery if you use Hamcrest matchers instead of Mockery matchers.
There is even a short hand for initializing mockery mocks, and support for counting Mockery expectations as assertions for better --strict mode support.
Check it out in the latest release of Etsy PHPUnit Extensions.
PHP CodeSniffer for Recently Changed Files
At Etsy we have been running PHP_CodeSniffer as part of our pre-commit. When we ran on the entire code base it took at least 5 minutes, which is too slow for the Etsy Continuous Deployment pipeline.
Immediately suggestions flew in to only run the sniffs on the changed files, but you can’t just define the set of changed files as files that have changed since the last run.
Why not? Because the set of changed files in the next run will not necessarily contain the files that changed. In fact, the clever programmer will watch the code sniffer find errors in his code and will then push the build button again to watch it turn green on an empty change set.
The elegant solution around this would be to maintain a dictionary of files that have violations in them. The Jenkins (or other CI system) job would run phpcs on changed files and the files that were listed in the dictionary of files that have violations. The job would then alter the dictionary to contain the files that are still failing, but maintaining dictionaries is too tedious to satisfy the minimums that we need.
We minimally need a Jenkins job that takes under 5 minutes to run phpcs on all files that have been recently changed without missing file that has a violation.
The solution, in Git:
-
Find a SHA that was recent enough to not include too many files, but old enough so that there has definitely been at least one change.
git log --format='%H' --since='-5 days ago' | tail -n1 -
Now get the list of the files changed since that SHA
git diff --name-only {$old_sha}..HEAD | sed '/^$/d' | uniq - Pass that list to
phpcs
Five days is long enough to account for very long weekends, but for the most part there is always someone reason to have a deployment per day. With the five day window we see runtimes of maybe a minute now.
This solution is not complete however because if you pass phpcs a list of files to run on, then the ignore patterns are ignored. In other words, you have to do the blacklisting of files, sub-directories, etc. on your own.
Ideally, I should probably look at PHP_CodeSniffer to fix this, but in the interim, I give you CSRunner.
CSRunner simply requires an implementation of the Scm interface, and whatever implementations of the Filter interface you so desire. I started with giving it a Git implementation of the Scm that let’s you pass a remote, branch, and a time window for changes, and a Filter implementation that allows you to list patterns as you would in your ruleset.xml to ignore named Blacklist. Rather then wasting so much effort on config files, formats, and parsers, the README gives you nice example of how to write a little script to utilize this code. Honestly, I feel you all can handle it.
Joe Kleinerman 10K
Race #1 of 9 for the guaranteed entry to the 2013 NYC ING Marathon, complete!
I did ok! Ran the 10K in 1:05:01
Just missed being under 1:05, argh!
Honestly, I wanted to do a little better, but I think a few things worked against me. First, I chickened out in the brisk morning and wore my sub-40F clothing to a nearly 50F race. Let’s say I was busy taking off clothing during the short race. Second, I chickened out on drinking water too close to the race. I feared having to stop in the middle of the race. I also was concerned that getting water during the race would cost me more than running towards dehydration. Due to these two shoulda-coulda-woulda-s racing through my head there were definitely times I psyched myself out and said to myself, “We don’t need to go fast. There is more training to do. Don’t risk injury on an insignificant race.”
Maybe next time I won’t psych myself out.
Developer Testing, Not Necessarily TDD… Revisited
In a previous article, I said
Test-Driven Development is a possible development workflow that incorporates Developer Testing, but Developer Testing does not necessarily only exist in Test-Driven Development (TDD).
Later I pulled together this graphic
 |
 |
I could equate all of the steps between TDD and Waterfall, but would gloss over the design phase. Then, one day, someone tweeted about TDD being a design process, and they were right.
The design step is missing because TDD in its entirety is a design process. Testing is a step in this process. TDD is not a process for testing. TDD is a process for designing.
That said, it is still valuable to practice Developer Testing no matter how you design, iterate, program, or whatever to achieve the desired end product.
