Re: Testing with concurrent sessions - Mailing list pgsql-hackers

Markus Wanner wrote:
>> I do want to expand the tests quite a bit -- do I work them all into
>> this same file, or how would I proceed?  I think I'll need about 20
>> more tests, but I don't want to get in the way of your work on the
>> framework which runs them.
> 
> Well, first of all, another piece of the missing manual: there are
> BaseTest and SyncTest classes. Those based on BaseTest runs within the
> event loop of the twisted framework, thus need to be written in the very
> same asynchronous fashion. Mostly calling async methods that return a
> Deferred object, on which you may addCallback() or addErrback(). See the
> fine twisted documentation, especially the part about "Low-Level
> Networking and Event Loop" here:
> 
> http://twistedmatrix.com/documents/current/core/howto/index.html

> I'm not sure about how to organize the tests and ongoing development of
> the framework. I've already broken the Postgres-R tests with dtester-0.0.

Hi,

sorry to butt in to the conversation, but I have spent some time
wrapping/refining the concepts in dtester, and the results are here:

http://git.wulczer.org/?p=twisted-psql.git;a=summary

It reqires Twisted and has been tested on Python 2.5 (should work on
2.6, no idea about 3.0). The program you use to run it - trial - should
come with your distro's Twisted packages. The tests don't start a server
or anything, so you need to have a PG instance running. To try it:

git clone git://wulczer.org/twisted-psql.git
cd twisted-psql # this is important, or Python won't find the modules
$EDITOR config.py # set the path to psql and connection details for PG
trial test.test_serialization_error
trial test.test_true_serialization

Both tests should pass, the latter being marked as an expectedFailure.
You can then look at test/*.py to see my (puny) attempt at having some
abstraction layer over the asynchronocity of the tests.

I borrowed the idea of wrapping a psql in a Twisted protocol and added a
Deferred interface around it, which made it possible to run tests with
trial: the Twisted unit testing framework.

As a developer of a failry large Python system based on Twisted, that
sports hundreds of trial-based tests, I very strongly recommend trial
for asynchronous unit testing. It handles lots of boring details, is
well maintained and Twisted itself is simply designed to do asynchronous
programming. As an added bonus, the runnning and reporting
infrastructure is already there, you just write the tests.

My code is very rough and lacks good error reporting, for instance
failed tests will probably result in a "test hung" and the need to
Ctrl+C, but that can be easily improved. A thing that would help
tremendously would be a real Twisted protocol that talks to PG on the
protocol level, not by parsing psql output (which is very clumsy and
error prone IMHO).

I found one such project:
http://www.jamwt.com/pgasync/
but it had some issues with committing (all my test programs were
exiting before PG got the final COMMIT, which resulted in the
impossibility to do anything) and it does too much things that Python PG
drivers like to do (like declaring a CURSOR for each query, bleah). A
good implementation would hugely improve the quality and robustness of
any such testsuite.

Cheers,
Jan
-- 
Jan Urbanski
GPG key ID: E583D7D2

ouden estin