Thread: New index structure with Open MP

Hi pgsql-general mailing list users,

I have a question related to the development of a new index structure.
I am a writing my masters thesis regarding index structures and the
possibility to parallize them.
I already found a post in the archives regarding Open MP, but my
question is somehow different. I am currently not aiming at a production
ready implementation, a prototype is sufficient. I already checked if I
can use a different database, e.g. Apache Derby or MySQL (because they
are already multithreaded), but it is rather complicated to extend them,
I think it is not intended to add new index structures within one of
them. So, long story short, PostgreSQL is optimal for development of a
new index structure and well documented (yeah! really! great! thanks a
lot for that!). I am not aiming for full parallelization, only some
parts of the algorithm regarding build, insert and search are going to
be extended by Open MP. E.g. I want to parallelize searching in multiple
pages by directing some portions to one thread and other portions to
another thread. Do you think that this small amount of parallelization
is possible? Or will there be complications with the used methods by the
buffer manager and so on? What do you think? What are your thoughts?

Greets, Yves W.

=?ISO-8859-15?Q?Yves_Wei=DFig?= <weissig@rbg.informatik.tu-darmstadt.de> writes:
> I am not aiming for full parallelization, only some
> parts of the algorithm regarding build, insert and search are going to
> be extended by Open MP. E.g. I want to parallelize searching in multiple
> pages by directing some portions to one thread and other portions to
> another thread. Do you think that this small amount of parallelization
> is possible? Or will there be complications with the used methods by the
> buffer manager and so on? What do you think? What are your thoughts?

The backend code is not designed for thread safety.  This is not a case
where "only a little bit" of parallelism is going to be safe.  It *will*
break.

            regards, tom lane

Hi,

thanks for the answer.
I understand that the backend is not thread safe, but it would be possible
to parallelize, let's say a big for-loop without breaking anything, or?

Greets, Yves W.

-----Original Message-----
From: Tom Lane [mailto:tgl@sss.pgh.pa.us]
Sent: Sunday, January 30, 2011 10:38 PM
To: weissig@rbg.informatik.tu-darmstadt.de
Cc: pgsql-general@postgresql.org
Subject: Re: [GENERAL] New index structure with Open MP

=?ISO-8859-15?Q?Yves_Wei=DFig?= <weissig@rbg.informatik.tu-darmstadt.de>
writes:
> I am not aiming for full parallelization, only some parts of the
> algorithm regarding build, insert and search are going to be extended
> by Open MP. E.g. I want to parallelize searching in multiple pages by
> directing some portions to one thread and other portions to another
> thread. Do you think that this small amount of parallelization is
> possible? Or will there be complications with the used methods by the
> buffer manager and so on? What do you think? What are your thoughts?

The backend code is not designed for thread safety.  This is not a case
where "only a little bit" of parallelism is going to be safe.  It *will*
break.

            regards, tom lane