Ideas for Google Summer of Code projects

Most of the stuff on the SimpleTest2 road map is incremental. Ideas for Google Summer of Code projects are of another kind. There're here to unleash and promote new crazy ideas to the project and to PHP's testing universe!

Of course this list is not exhaustive: if you have another itch to scratch, feel free to add your wishes on the mailing-list.

Enable JavaScript inside SimpleTest browser.

This would be a MASSIVE feature to have. We're probably talking about integrating a C-based or Java-based JavaScript engine. The Java folks have HttpUnit already : some stuff could be derived from there. Demonstrating the feasibility alone would be a real step forward. The hard work is the DOM emulation.

Add layer testing.

When using an MVC framework is to support bypassing the webserver, and interacting with the framework at the level of Request and Response objects. It would be nice to see if we could develop an interface standard for the HTTP layer which would allow SimpleTest to work cleanly with different frameworks. On the other hand, maybe there already is enough of an interface in the standard PHP superglobals. Symfony with its lime test framework works in exactly this manner.

Make parsers switchable and change the HTML Parser.

Replace the html parser with tidylib. It's super-fast compared to parsing in PHP. If there are worries about adding a dependency, then maybe make it an optional plug-in instead.

Web Form Fuzzer.

Fuzz testing is a software testing technique that provides random data ("fuzz") to the inputs of a program. If the program fails (for example, by crashing, or by failing built-in code assertions), the defects can be noted.
A Web Form Fuzzer would be built on top of the basic WebTestCase api, enabling developers to point a test case at a target URL and have it introspect form inputs and post actions. The fuzzer would generate random data strings, populate the input fields, and repeatedly post the forms, looking for errors, validation responses, and HTTP status codes that reflect how the site handled a combination of well-formed and malformed input. This would have the potential to completely wreck some websites, but would be great for testing the robustness and edge case behavior of CMS's and other online administration tools.

Standalone notifying tool

Based on rules, this notifying tool detects a file change within a particular directory (maybe with php-inotifytools : such a monitor could learn how your code works by observing it being tested. For example, call graphing via static analysis might work, as would running Xdebug code coverage while running the tests and monitoring the code executed after each test method, as would some sort of DBG-derived client that received notifications from Xdebug while the code was being executed like a debugging client. We'd have "rings" of functionality. The first ring is the tests that exercises the particular method/function you're working with by a direct call. Then start working rings out based on how far away the test method is when that code is called. These rings would then be executed on a save, starting with the inner most. Any failure would stop further tests cause a system alert to something like Growl or Snarl or any Reporter that you drop in.

PDT (Eclipse) : implement PHPUnit hooks

There are somme hooks inside PDT implemented for PHPUnit. But it seems there're unused since PHPUnit changed too much recently. SimpleTest could use them as well : the SimpleTest Eclipse plugin could then become the PDT Test plugin while fixing those hooks.

PDT (Eclipse) : debug while unit testing

The GUI for Eclipse recently caught up with PDT : debugging while running a SimpleTest test case is finally doable.

Creative IDE integration

For example, running a subset of tests after each edit.

Firefox plugin to create SimpleTest file on the fly.

The same way Selenium IDE currently works.

Komodo integration.

Integration with monitoring tools such as Nagios.

Browser-based

... and Unit testing-related ideas for GSOC2008.

Also of interest, integration with third-party tools.

Google Summer of Code 2008.