| assertTrue($x) |
Fail if $x is false |
$this->assertTrue(1 == true);
|
| assertFalse($x) |
Fail if $x is true |
$this->assertFalse(1 === true);
|
| assertNull($x) |
Fail if $x is set |
$variable = null;
$this->assertNull($variable);
|
| assertNotNull($x) |
Fail if $x not set |
$variable = 'something but not null';
$this->assertNotNull($variable);
|
| assertIsA($x, $t) |
Fail if $x is not the class or type $t |
$tim = new Person;
$this->assertIsA($tim, 'Person');
|
| assertNotA($x, $t) |
Fail if $x is of the class or type $t |
$tim = new Person;
$this->assertNotA($tim, 'Animal'); // or maybe I am? o_O
|
| assertEqual($x, $y) |
Fail if $x == $y is false |
$fahrenheit = 50;
$celsius = (5/9)*($fahrenheit-32);
$this->assertEqual(10, $celsius);
|
| assertNotEqual($x, $y) |
Fail if $x == $y is true |
$this->assertNotEqual('5', 5); // fails, they are equal but not identical
|
| assertWithinMargin($x, $y, $m) |
Fail if abs($x - $y) < $m is false |
$this->assertWithinMargin(10, 50, 60); // passes
$this->assertWithinMargin(10, 50, 30); // fails
|
| assertOutsideMargin($x, $y, $m) |
Fail if abs($x - $y) < $m is true |
$this->assertOutsideMargin(10, 50, 60); // fails
$this->assertOutsideMargin(10, 50, 30); // true
|
| assertIdentical($x, $y) |
Fail if $x == $y is false or a type mismatch |
$this->assertIdentical(0, false); // will fail since 0 is not false
|
| assertNotIdentical($x, $y) |
Fail if $x == $y is true and types match |
$this->assertIdentical(10, 100/10*5 - 40);
|
| assertReference($x, $y) |
Fail unless $x and $y are the same variable |
$a = 5;
$b =& $a;
$this->assertReference($a, $b);
|
| assertClone($x, $y) |
Fail unless $x and $y are identical copies, that means they are identical but not referenced |
$a = 5;
$b = 5;
$this->assertClone($a, $b); // passes, 5 equals 5, but $a and $b are not references to each other
|
| assertPattern($p, $x) |
Fail unless the regex $p matches $x |
$this->assertPattern('/hello/i', 'Hello world');
|
| assertNoPattern($p, $x) |
Fail if the regex $p matches $x |
$this->assertNoPattern('/heppo/i', 'Hello world'); // passes
|
| expectError($x) |
Swallows any upcoming matching error |
|
| assert($e) |
Fail on failed expectation object $e; use SimpleTest's built in expectation objects to have fun here |
<?php
class TestOfNetworking extends UnitTestCase {
...
function assertValidIp($ip, $message = '%s') {
$this->assert(new ValidIp(), $ip, $message); // uses validIp expectation
}
function testGetValidIp() {
$server = &new Server();
$this->assertValidIp(
$server->getIp(),
'Server IP address->%s');
}
}
?>
|
So, with the custom assert() function, mentioned last in the table, you can build your own assertions keeping your code really clean. The others should be straightforward. If not, head on over to http://simpletest.org and read the documentation there.
When any of these assertions fail, you will be presented an error message telling you which test failed, on which line and what the error is. What if you want to supply your own error messages - for example to better mark an often failing test? You can do that in simpletest pretty easily. Just append your custom message as the last parameter to the assertion:
$this->assertTrue(1, 'This should pass');
You will notice that it replaces the automatic error message. If you want to embed your custom message within the automatic one, use %s:
$this->assertTrue(1, 'This should pass: %s');
1.2 A quick SimpleTest example
Okay so before we are going to jump onto the cake test wagon let's look at a real world simpletest example. That way it will be easier for us to deal with cake's testsuite later.
So here is the problem:
You want to calculate the costs for a given trip (to a cakefest). You get: $flightPrice, $hotelCosts and expenses for food and drinks ($otherExpenses). You know you will need more money since gwoo (the president of the Cake Software foundation) will come along persuading you to do sling shot (and yes we had a lot of fun at Cakefest in Orlando :]) although you are already a little drunk. So we take on a 10% buffer that we add to the sum. Our total expenses are:
Formula: $total = 110% of ($flightPrice + $hotelCosts + $otherExpenses)
So, we might end up with:
<?php
require_once('simpletest/unit_tester.php');
// install simpletest into /vendors after you downloaded it from http://simpletest.org
require_once('simpletest/reporter.php');
define('BUFFER_RATE',
0.1);
function calcTravelExpenses
($flightPrice,
$hotelCosts,
$otherExpenses) {
$sum =
$flightPrice +
$hotelCosts +
$otherExpenses;
$sum =
(BUFFER_RATE
) *
$sum;
}
class TestOfTravelExpenses
extends UnitTestCase
{
function testExceededTravelExpenses
() {
$this->
assertEqual(1320, calcTravelExpenses
(600,
400,
200),
'Okay so this is a pretty exciting test %s message end');
}
}
$test =
&new TestOfTravelExpenses
();
$test->
run(new HtmlReporter
());
?>
When you run this you see you get a red bar with the message that "Integer" differs from "NUll". So 1320 is obviously an integer and the output of our function is NULL. Uh oh, okay, so there is no return statement! Let's add it:
function calcTravelExpenses($flightPrice, $hotelCosts, $otherExpenses) {
$sum = $flightPrice + $hotelCosts + $otherExpenses;
$sum = (BUFFER_RATE) * $sum;
return $sum; // here
}
Now the value is not quite right. Ah okay, because we are taking 10% of the sum instead of 110%:
$sum = (1+BUFFER_RATE) * $sum;
Green bar! Now that's a great feeling right? We can do all sorts of crazy refactorings now to our little function and still we will see if it works or not. And all of that automagically! Wohoo! Right, so what are the benefits of all this?
1.3 Benefits of Unit Testing
1. Unit Testing allows us to refactor our code later with real confidence. If we break something and have plenty of tests there should be a test failing, we fix it up and the entire system should work again (that means all units independently of each other). Great!
2. It forces you to think about what your code is really supposed to do. You finally get rid of the script kiddy attitude that hacks something together fast and does not even lose a second on thinking if his unit even returns an integer or if there could not be a division by zero error. I am not sure how it happens, but you will think about the edge cases, the most important cases, automatically. It's so difficult to think about them without automated tests, but with UT you approach the problem from a more theoretical and mathematical side.
3. Automatic testing is faster than browser testing. Period.
4. UT provides living documentation. If you develop in a team with several people and someone wrote a unit without phpdoc comments and you have no idea what it does, you can frequent the tests for it. Understanding code from tests is ultra easy and fast and most of the time better than any discussion. You could even picture the code in your head (if you are smart enough) just looking at the tests.
5. It helps you separate interface from implementation. Code that uses your code works with your interface, that is the unit name and the parameter signature and your return type. If you don't change that you can do what you want in the unit implementation. It's so great because it gives you so much freedom. Also as long as the original tests still run, you ensure backwards compatibility of your code, too.
6. Make the CakePHP Core Team happy. Many people submit tickets to CakePHP to make us aware of bugs. That's a good thing. However, if more people submitted tests alongside their bugreports that would help even more.
There are plenty of other benefits..
1.4 Limitations of Unit Testing
1. UT as such does not prove there are no errors in your system. It is not a theoretical / mathematical prove that a particular unit is bug free. Yes it shows the presence of errors if you have good and enough tests. However, it does not show the absence of errors.
2. Besides that, unit testing tests the units of your system independently of each other (for the most part). So if your system suffers from performance problems or integration problems, unit testing will not catch them. Also it will not prove that your system is not vulnerable to any security attack from the outside.
3. What's more? Well, many people don't implement unit testing yet, because it takes a rigorous amount of discipline to do it consistently. Especially in a team environment with clients paying, tight deadlines and all sorts of other interruptions, testing seems to be the first thing to cut on. However, in the long term it will prove to be much more productive to do it. The drawback stays, though: You write extra code that does not add any "real" features to your application. Don't get me wrong - I love to do it, but I can partly understand the clients, too. Would be cool if you could throw in your two bits on this one.
Conclusion: Do it! No I mean try it out. And if you only implement ten tests in your application, you are still much better off than without any tests.
1.5 Test-driven Development
So, when do you write the tests? From what you have read so far, you must have the idea that you write the tests after you wrote the code. However, with that "interface-over-implementation" and that "think-about-it-before-you-start-it" benefits, wouldn't it be cool to write the tests before you write the code? Test-driven Development - also known as TDD - does that.
Essentially TDD is a software development technique consisting of short iterations where new tests covering the desired improvement or new functionality are written first. Then you implement the production code necessary to pass the tests. TDD helps a lot, to make your code design nice and to accommodate changes you refactor.
The availability of tests before actual development ensures rapid feedback after any change. Remember that green-bar-means-you-are-done and red-bar-means-work? TDD ultimately boils down to those.
TDD is actually a method of designing your software instead of testing it. Why? Because with TDD you REALLY think about the stuff you do before you do it. If you have a problem you cannot tackle, you write the simplest possible test, make it work, and go from there. As you write more tests and as you think about how people should use your code (ie, what your interface shall look like), you design your software well automatically. Yeah, it's not always that easy, but you get the idea.
In TDD you have a rough cycle that should take just minutes, if not seconds:
Add a test -> see it failing -> make it work -> see the green bar -> refactor -> still see the green bar - > add a test -> see it failing -> ...
1.6 Mock Objects
Mock Objects are objects that simulate real objects that would normally be difficult to construct or time consuming to set up for a test. The most common use of mock objects is the mocking of a database connection object.
Setting up a test database at the start of each test would slow testing to a crawl and would require the installation of the database engine and test data on the test machine. If the connection can be simulated you can return data of your choosing. By that you not only win on the pragmatics of testing, but can also feed your code spurious data to see how it responds. You could simulate databases being down or other extremes without having to create a broken database for real. In other words, you get greater control of the test environment.
However, the mock objects not only play a part (by supplying chosen return values on demand) they are also sensitive to the messages sent to them (via expectations). By setting expected parameters for a method call they act as a guard that the calls upon them are made correctly. If expectations are not met they save you the effort of writing a failed test assertion by performing that duty on our behalf.
This can be very useful, because with standard Unit Testing you just test the interfaces of your objects. Mock Objects give you a means to test the inner implementation of them.
1.7 Resources and Further Reading
Process is at least as important as tools. The type of process that makes the heaviest use of a developer's testing tool is of course Extreme Programming. You should also read about Agile Methodologies in general. If you want to read up on test-driven Development, please do so here.
The original Unit Test Framework was JUnit. Most people writing their own test tools seem to be cloning it in one way or the other. PHP Unit is the XUnit ambassador for the PHP world. Together with Simpletest it forms the top testing framework in the php world.
Wrap Up
By now you should have an idea of what you can get out of Unit Testing and automated tests in general. I hope I have raised some questions or made controversial statements. As we discuss this article, some good things can be added to it I am sure.
In the next part of the series we will look at Cake's Testsuite.
Posted on 3/5/08 by
Felix Geisendörfer
Hey folks,
for some of the projects Tim and I recently got involved with I noticed that I needed a way to share a set of test data with the other people working on the project. The result is a simple shell leveraging CakePHP 1.2's fixtures. Usually those are used for unit testing your Models and Behaviors, but they also do a great job at sharing a common set of data within a project.
So if you're interested to try it, check out the project page and let me know what you think.
-- Felix Geisendörfer aka the_undefined
Posted on 1/5/08 by
Felix Geisendörfer
Hey folks,
I don't know how often I had to discuss this with people over the past months, so let me say this once and for all:
XHTML is a joke. It is an elitist trend against the very nature of the web and offers no advantages over HTML 4.01 strict!
This is an old old discussion and there are great resources that list all of the problems associated with using XHTML. All I want to address in this post are two problems I have with XHTML:
1) Strict rendering is a STUPID idea
I don't know if I am the only one that realizes this. But if all your favourite web sites were to be converted to XHTML strict most of them would be unusable half of the time because of tiny errors in views. XHTML promotes that the slightest mistake in your markup will cause your page to NOT RENDER!
To me this is the most ridiculous concept ever. The view / presentation is the least important layer in any application and XHTML says it should be able to completely crash your page. Give me a break, but this is madness.
[Edit: 2008-05-02] When I say that the view is the least important layer than I mean that it should be most the forgiving layer because it does no business critical data processing.
Publishing content on the web should be easy for everybody. I am against anything raising the bar in that regard. Of course I wish MySpace never happened, but millions of people absolutely love the idea of expressing themselves in the most ridiculous ways. Heck, if I never had gotten into the web industry you can bet that I would have a 5mb MySpace page that brings any browser to its knees.
So let us come up with better standards, but please make them forgiving about mistakes within reasonable boundaries. The main reason we all use PHP (or any other dynamically typed language) is that we hate strict boundaries. We love flexibility in any form, and I strongly believe that flexible and forgiving standards are far superior in terms of enabling innovation compared to strict standards like XHTML.
2) Presentation agnostic XHTML is an ILLUSION
The biggest advantage people see in XHTML is that it is supposedly better at separating content from presentation. That is BS. XHTML pretty much supports the same set elements as HTML, including strictly presentational ones. But more importantly: If you ever tried to completely change the design of an XHTML page I know that you cheated and changed the markup.
[Edit: 2008-05-02] The above is meant in the context of real world situations with dynamic content. CSS Zen Garden and similar sites are not realistic web environments. See comment below.
So if you are under the impression that by using XHTML your pages automatically become presentation agnostic data sources that can be aggregated you are hallucinating. Microformats and web scraping is what you should look into.
Conclusion: I think XHTML did a ton of good for the web. It gave people a way to draw a line between traditional table driven HTML layouts and modern XHTML / CSS approaches. It spawned a wave of support for semantic markup and web standards. But looking at XHTML as a format by itself I see no advantages in it. I have big hopes for HTML5. Meanwhile I feel very happy with the feature set and philosophy provided by HTML 4.01 and will continue to use it.
Looking forward to feedback on this.
-- Felix Geisendörfer aka the_undefined
Posted on 27/4/08 by
Tim Koschützki
Hey folks,
the last week I have been dealing with an implementation of code coverage analysis for the upcoming release 1.2 of CakePHP. For those of you who do not yet know what code coverage is, here is a short excerpt from Wikipedia:
Code coverage is a measure used in software testing. It describes the degree to which the source code of a program has been tested. It is a form of testing that inspects the code directly and is therefore a form of white box testing.
There are several different kinds of criteria to code coverage. The two most important ones are line coverage (or statement coverage as wikipedia puts it) and path coverage. Line coverage basically analyzes if each line of the subject-under-test has been executed during the test. Path coverage examines if every possible route through the code has been executed. Since I rely on Xdebug for the implementation we are talking line coverage here.
So how is it going to work? Pretty simple actually. Whenever you run a CakePHP test case Cake assembles information in the background about which lines of your subject-under-test are called. Then it will display all code lines from the subject-under-test and mark those that were "covered" (ie executed by the testcase) and which were not. Lines that play no role for the code coverage percentage, such as multiline comments, will be displayed in a different color with less contrast.
How could an example report look like? Here you go:
I have also implemented a diff-style report. It just lists all the lines that are not covered by your tests and then seven lines before and after each of those. With that view you can really focus on the uncovered lines, but see enough lines around them in order to be able to write a reasonable test case to get the uncovered lines covered.
This is what the diff view looks like for the cakephp core router.test.php:
Within the dev team we are currently discussing these two views. If you favor one of these, can recommend another one or just want to voice your opinion on the looks of them, please do so by commenting on this post.
For your information: Code coverage analysis currently only works for separate test cases only. However, the plan for this week is to get aggregate code coverage working. This means that your group tests are analyzed and the coverage percentage is displayed for each of the test subjects involved.
This is a very valuable tool when developing tests for your cakephp application. Although we are talking line coverage here and not path coverage it is still a pretty good indicator of how thorough your tests are. It might as well encourage some more people to start writing tests if they see a neat report after they have done so. :]
-- Tim Koschuetzki aka DarkAngelBGE
William: thanks, fixed ..
Hi Felix,
The Upgra ..
Maybe you can write so ..
This is very nice plug ..
Nice article!
Somet ..
That's fine ognian, th ..
Really nice work! I wa ..
Thanks didn't quite un ..