Re: MySQL file system - Mailing list pgsql-general

This idea is such a popular (even old) one that Oracle developed it for 8i --
IFS.  Yep, AS/400 has had it forever, and BeOS is another example.  Informix has
had its DataBlades for years, as well.  In fact, Reiser-FS is an FS implemented
on a DB, albeit probably not a SQL DB.  AIX's LVM and JFS is extent/DB-based, as
well. Let's see now, why would all those guys do that?  (Now, some of those that
aren't SQL-based probably won't allow SQL queries on files, so just think about
those that do, for a minute)....

Rather than asking why, a far better question is why not?  There is SO much
functionality to be gained here that it's silly to ask why.  At a higher level,
treating BLOBs as files and as DB entries simultaneously has so many uses, that
one has trouble answering the question properly without the puzzled stare back
at the questioner.  Again, look at the above list, particularly at AS/400 -- the
entire OS's FS sits on top of DB/2!

For example, think how easy dynamically generated web sites could access online
catalog information, with all those JPEG's, GIFs, PNGs, HTML files, Text files,
.PDF's, etc., both in the DB and in the FS.  This would be so much easier to
maintain, when you have webmasters, web designers, artists, programmers,
sysadmins, dba's, etc., all trying to manage a big, dynamic, graphics-rich web
site.  Who cares if the FS is a bit slow, as long as it's not too slow?  That's
not the point, anyway.

The point is easy access to data:  asset management, version control, the
ability to access the same data as a file and as a BLOB simultaneously, the
ability to replicate easier, the ability to use more tools on the same info,
etc.  It's not for speed, per se; instead, it's for accessibility.

Think about this issue.  You have some already compiled text-based program that
works on binary files, but not on databases -- it was simply never designed into
the program.  How are you going to get your graphics BLOBs into that program?
Oh yeah, let's write another program to transform our data into files, first,
then after processing delete them in some cleanup routine....  Why?  If you have
a DB'ed FS, then file data can simultaneously have two views -- one for the DB
and one as an FS.  (You can easily reverse the scenario.)  Not only does this
save time and disk space; it saves you from having to pay for the most expensive
element of all -- programmer time.

BTW, once this FS-on-a-DB concept really sinks in, imagine how tightly
integrated Linux/Unix apps could be written.  Imagine if a bunch of GPL'ed
software started coding for this and used this as a means to exchange data, all
using a common set of libraries.  You could get to the point of uniting files,
BLOBs, data of all sorts, IPC, version control, etc., all under one umbrella,
especially if XML was the means data was exchanged.  Heck, distributed
authentication, file access, data access, etc., could be improved greatly.
Well, this paragraph sounds like flame bait, but really consider the
ramifications.  Also, read the next paragraph....

Something like this *has* existed for Postgres for a long time -- PGFS, by Brian
Bartholomew.  It's even supposedly matured with age.  Unfortunately, I cannot
get to http://www.wv.com/ (Working Version's main site).  Working Version is a
version control system that keeps old versions of files around in the FS.  It
uses PG as the back-end DB and lets you mount it like another FS.  It's
supposedly an awesome system, but where is it?  It's not some clunky korbit
thingy, either.  (If someone can find it, please let me know by email, if
possible.)

The only thing I can find on this is from a Google search, which caches
everything but the actual software:

http://www.google.com/search?q=pgfs+postgres&num=100&hl=en&lr=lang_en&newwindow=1&safe=active

Also, there is the Perl-FS that can be transformed into something like PGFS:
http://www.assurdo.com/perlfs/  It allows you to write Perl code that can mount
various protocols or data types as an FS, in user space.  (One example is the
ability to mount FTP sites, BTW.)

Instead of ridiculing something you've never tried, consider that MySQL-FS,
Oracle (IFS), Informix (DataBlades), AS/400 (DB/2), BeOS, and Reiser-FS are
doing this today.  Do you want to be left behind and let them tell us what it's
good for?  Or, do we want this for PG?  (Reiser-FS, BTW, is FASTER than ext2,
but has no SQL hooks).

There were many posts on this on slashdot:
    http://slashdot.org/article.pl?sid=01/01/16/1855253&mode=thread
    (I wrote some comments here, as well, just look for mikehoskins)

I, for one, want to see this succeed for MySQL, PostgreSQL, msql, etc.  It's an
awesome feature that doesn't need to be speedy because it can save HUMANS time.

The question really is, "When do we want to catch up to everyone else?"  We are
always moving to higher levels of abstraction, anyway, so it's just a matter of
time.  PG should participate.


Adam Lang wrote:

> I wasn't following the thread too closely, but database for a filesystem has
> been done.  BeOS uses a database for a filesystem as well as AS/400 and
> Mainframes.
>
> Adam Lang
> Systems Engineer
> Rutgers Casualty Insurance Company
> http://www.rutgersinsurance.com
> ----- Original Message -----
> From: "Alfred Perlstein" <bright@wintelcom.net>
> To: "Robert D. Nelson" <RDNELSON@co.centre.pa.us>
> Cc: "Joseph Shraibman" <jks@selectacast.net>; "Karl DeBisschop"
> <karl@debisschop.net>; "Ned Lilly" <ned@greatbridge.com>; "PostgreSQL
> General" <pgsql-general@postgresql.org>
> Sent: Wednesday, January 17, 2001 12:23 PM
> Subject: Re: [GENERAL] MySQL file system
>
> > * Robert D. Nelson <RDNELSON@co.centre.pa.us> [010117 05:17] wrote:
> > > >Raw disk access allows:
> > >
> > > If I'm correct, mysql is providing a filesystem, not a way to access raw
> > > disk, like Oracle does. Huge difference there - with a filesystem, you
> have
> > > overhead of FS *and* SQL at the same time.
> >
> > Oh, so it's sort of like /proc for mysql?
> >
> > What a terrible waste of time and resources. :(
> >
> > --
> > -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org]
> > "I have the heart of a child; I keep it in a jar on my desk."