You are currently browsing the Test Me posts tagged: tests


New Addition to CSRunner: php -l

Speed is addictive.

After speeding up PHP Code Sniffer runs, the slowest job for Etsy was running php -l on all the files.

So, in short, the latest release of CSRunner, version 0.2.1, will execute lint on the recently changed files.

You can also run the lint command separately, but why would you want to do this instead of your precious find piped to a php -l script?

Because this command outputs to checkstyle format!

Which means if you are Jenkins and using the Jenkins Checkstyle Plugin, you can add the resulting checkstyle xml to your list of Checkstyle results. Voila!

Enjoy!

Thoughts on Accessibility with Regards to Testing

In Alternative Thoughts on Sniffs for Useless Method Overriding I discussed the discovery of a horrible dance that some developers will do just to test private functions:

  1. See a private method you want to test in isolation
  2. Upgrade the private to protected
  3. Create a child class implementation of this class in the TestCase file
  4. Override any protected methods, that were actually intended to be private methods and make them now public
  5. Test the new child class instead of the actual class

This violates the only one class per test file rule, as discussed in PHPUnit Coding Standard: Classes

Many claim that they do this because they do not want the methods to be part of the public API, but need to make it protected so they can do this hack. Yet those who argue for this so called solution have nothing to reconcile the fact that they have removed the ability to distinguish between methods that should be override-able by child implementations and those that should not.

In all honesty, if you have functionality that you would prefer not to be in the public API of one object, but it is sufficiently complex to warrant it’s own battery of tests, you should take that as a suggestion to pull that functionality out into it’s own object.

It is a pity to hide something so complex in a private function, and tempt others into exercising copy/paste coding practices to use it.

If you are using PHP 5.3, then you can do another cheat that will let you avoid the number of class violations, and let you keep using the private keyword. Sebastian Bergmann discusses this in Testing Your Privates.

Don’t jump the gun. This is still not license to blindly make things private. Even Sebastian repeats in the post and in the comments:

Just because the testing of protected and private attributes and methods is possible does not mean that this is a “good thing”.

So why am I even talking about these naughty ways to test private methods?

Because I cannot necessarily stop you!

So let me offer an alternative to all of this that should help communicate intent in your code, let you use all the accessibility levels, and keep you from re-hashing reflection calls in your test methods.

Today I put out a new release of the Etsy PHPUnit Extensions. This new release includes PHPUnit_Extensions_Helper_AccessibleObject.

PHPUnit_Extensions_Helper_AccessibleObject is a wrapper for classes which contain private or protected methods that you would like to test in isolation, for some bizarre reason. But, you may only call private and protected methods of the wrapped object that are annotated with @accessibleForTesting.

Why the special @accessibleForTesting annotation on methods in my production code?

With the @accessibleForTesting you will be able to communicate with other developers that while this code is private or protected you intend to test the logic in it. Also, we can easily comb through the code with a static analyzer to find spots in our code that we might want to consider extracting into more reusable places, like another class. Furthermore, we can also sniff for usages of Reflection.* in our tests and useless method overrides, then double back to ensure we move everything to this AccessibleObject method for testing private and protected methods so that we can go back to the loveliness mentioned in the first half of this paragraph.

If you were making the argument for doing the horrible dance to test private methods mentioned in the beginning of this article, then you can certainly get onboard with being more descriptive about your intent with @accessibleForTesting.

For examples, usage, and code: https://github.com/etsy/phpunit-extensions/wiki/Helpers

P.S. This is still naughty! But it’s a compromise I think I might be able to live with.

Mockery Test Case for PHPUnit

Mockery is a wonderful PHP mock object framework. It reminds me of Mockito for Java.

Mockery is far more fluent than MockObjects, the mock object framework packaged with PHPUnit. Mockery and MockObjects can work in the same environment, independent of one another.

Since Mockery is a free standing library, it can be used in PHPUnit, but you need to make sure you either have a loader that supports Mockery or use a bootstrap.php to use the loader packaged with Mockery. You will also need to consider using the TestListener packaged with Mockery to avoid having to remember to call Mockery::close() in tearDown for each TestCase, and a TestListener requires a PHPUnit XML configuration file.

All of this is do-able, but just not fun.

Now there is another alternative:

PHPUnit_Extensions_Mockery_TestCase

Using this new test case eliminates the bootstrap.php, TestListener with XML configuration, and explicit static calls to Mockery if you use Hamcrest matchers instead of Mockery matchers.

There is even a short hand for initializing mockery mocks, and support for counting Mockery expectations as assertions for better --strict mode support.

Check it out in the latest release of Etsy PHPUnit Extensions.

Developer Testing, Not Necessarily TDD… Revisited

In a previous article, I said

Test-Driven Development is a possible development workflow that incorporates Developer Testing, but Developer Testing does not necessarily only exist in Test-Driven Development (TDD).

Later I pulled together this graphic

I could equate all of the steps between TDD and Waterfall, but would gloss over the design phase. Then, one day, someone tweeted about TDD being a design process, and they were right.

The design step is missing because TDD in its entirety is a design process. Testing is a step in this process. TDD is not a process for testing. TDD is a process for designing.

That said, it is still valuable to practice Developer Testing no matter how you design, iterate, program, or whatever to achieve the desired end product.

Developer Testing, Not Necessarily TDD

During most of my tenure at Google, I taught the Developer Testing course to nearly all the engineering nooglers in the NYC office, and as well as some other east coast offices and engineers visiting from Australia.

At Etsy, I spend a fair amount of time actually thinking about developer testing in addition to developer happiness and ways to increase developer throughput.

Test-Driven Development is a possible development workflow that incorporates Developer Testing, but Developer Testing does not necessarily only exist in Test-Driven Development (TDD).

TDD is a development style with a development cycle:

  1. Add a tests for the new functionality
  2. Run all tests and see that the new tests FAIL
  3. Implement the new functionality
  4. Run all tests and see that the new tests PASS
  5. Refactor code
  6. Repeat

Thinking about it, it’s kind of like a re-wording for the Waterfall Development Model

But I don’t think that is the issue.

Recently someone asked me about TDD, and I spoke of my hesitancy towards the idea. I told him that the whole notion of having to test first was too harsh.

He, being a believer in TDD being the way to go, was skeptical that relaxing the test first constraint would be a bad thing. But the reality is, those who can test first and work well with test first will test first no matter what, but those who cannot test first because they are overwhelmed, inexperienced, or just plan do not think that way, they will likely not test or not produce.

So he took it further, and tried to argue about inside-out versus outside-in. The notion being that if you code the implementation first and then the test (inside-out) then you will code the implementation the easiest way possible for you and the test will be difficult because it is playing second fiddle. If you code the test first and the implementation second (outside-in) then you will implement the tests against an easy to use interface, and the implementation will be written to fulfill the easy to use interface.

I honestly do see the point on inside-out versus outside-in, but you can achieve the simple to use interfaces while testing second. You just have to remember one thing:

Tests should always be treated like every other consumer of the subject under test.

In short for now, that means that if you find yourself doing something tricky in your test to work around the interface of the the subject under test (SUT), you should fix the interface of the SUT and not do something tricky in your test. Your test is a consumer just like any other object that consumes the SUT in your production code, and if it is difficult to consume it in a test, then it will be difficult to consume it in production.

Getting back to the real point at hand, Developer Testing versus TDD. It is really important that as a developer you practice some sort of developer testing.

It could be manually testing your code, but if you manually test just remember that you should write down your manual tests. Writing out the tests is a good form of communication. It will allow other developers to know what they should be testing for when they begin to play in that area of the code.

It could be an automated functional test, which is like writing down a manual test case only better because it is executable. The only warning on these is that they have many points of failure. If you decide to incorporate these into a continuous deploy environment, you will want to consider either running them as probers or alerts through something like nagios, and not as red/green tests.

It could be xUnit tests, which can be either integration tests or unit tests. Integration tests give piece of mind that the actual implementations of collaborating object work together, and unit tests allow for small, fast, nimble, deterministic tests that ensure that the logic in the individual functions are correct while testing against the interface of the collaborators. Integration tests should generally be treated with the same care as automated functional tests, as discussed above. Unit tests are great for being red/green tests in a deployment pipeline.

We could dive into more about all these ways of testing, but the main point being: as a developer it is important that you test. If you write code, throw the code over the proverbial wall, wait for a response from a tester, fix, and repeat you will waste a lot of time. You will really be doing Waterfall Development. The other thing you will lose is the opportunity to communicate what you have accomplished and how you interpreted the requirements. You will miss out on executable documentation, whether by human (manual) or machine (automated).

Do a favor for yourself and all your peers, subscribe to developer testing. There are many ways to do it.

PHPNW11

Slides for my talk “Are Your Tests Really Helping You?” are now available on Slide Share.

As presented at PHPNW11 in Manchester, UK.

PHPUnit Coding Standard: https://github.com/elblinkin/PHPUnit-CodeSniffer
(P.S. I owe tons of documentation and a few “removal of assumptions”)

BTW: #PHPNW11ROCKS

PHPNW 2011

If you can make it to Manchester, UK for October 7-9, 2011, then you should consider coming to PHPNW 2011. Buy tickets here.

The schedule looks fantastic. Sebastian Bergmann and thePHP.cc will be there. Also, from my limited experience with this conference so far, Lorna Mitchell and crew run a very tight ship, which has left me even more impressed and excited.

I am very excited to say I will be also be speaking at PHPNW

Are Your Tests Really Helping?

Developer testing can reduce debug time, serve as executable documentation, build confidence, expose questionable patterns running rampant in your code, and in general, increase the speed of development and deployment. Tests can also cost you time, sanity, and agility.

This session will not be the same old re-hash of the Misko Hevry talk on testability. Instead of a talk that is generic, syntactically translated from Java to PHP, and neglectful the major coding patterns prevalent in existing PHP 5 code bases, all of which results in the majority of the audience as un-sold, we will look at coding and testing patterns inspired by a real PHP project. We will also discuss how to identify patterns and make small adjustments where testing is and is not helping. The end result will be a toolbox of habits we can use to improve testability and forward momentum in development.

I do honestly appreciate the gold standard talks on “Clean Code” (“Thank you, sir”), but those talks are nearly four years, if not more, old.

PHP is the driver of websites that touch millions, if not hundreds of millions, of users, and it is about time everyone stops doing the same-old hand-me-down presentations outside of a new hire training or a college classroom, and I’m tired of it.

I will try my best to give a great talk that can leave the crowd excited to go home and re-assess what is and isn’t working in the production of his PHP product and where to go from there.

Now, back to writing a less promotional blog post.