Re: pgsql: Replace our traditional initial-catalog-data format with abette - Mailing list pgsql-committers

Seems, gaur animal doesn't too long shell command. Could "find | xargs" 
help here?

cp ./*.h 
'/home/bfarm/bf-data/HEAD/pgsql.build/tmp_install/home/bfarm/bf-data/HEAD/inst/include/postgresql/server'/ 
|| exit; \
    chmod 644 
'/home/bfarm/bf-data/HEAD/pgsql.build/tmp_install/home/bfarm/bf-data/HEAD/inst/include/postgresql/server'/*.h 
  || exit; \
    for dir in access bootstrap catalog commands common datatype executor 
fe_utils foreign lib libpq mb nodes optimizer parser postmaster regex 
replication rewrite statistics storage tcop snowball snowball/libstemmer 
tsearch tsearch/dicts utils port port/atomics port/win32 port/win32_msvc 
port/win32_msvc/sys port/win32/arpa port/win32/netinet port/win32/sys 
portability; do \
      cp ./$dir/*.h 
'/home/bfarm/bf-data/HEAD/pgsql.build/tmp_install/home/bfarm/bf-data/HEAD/inst/include/postgresql/server'/$dir/ 
|| exit; \
      chmod 644 
'/home/bfarm/bf-data/HEAD/pgsql.build/tmp_install/home/bfarm/bf-data/HEAD/inst/include/postgresql/server'/$dir/*.h 
  || exit; \
    done
/bin/sh[5]: /usr/bin/chmod: The parameter list is too long.

https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=gaur&dt=2018-04-08%2017%3A20%3A57

Tom Lane wrote:
> Replace our traditional initial-catalog-data format with a better design.
> 
> Historically, the initial catalog data to be installed during bootstrap
> has been written in DATA() lines in the catalog header files.  This had
> lots of disadvantages: the format was badly underdocumented, it was
> very difficult to edit the data in any mechanized way, and due to the
> lack of any abstraction the data was verbose, hard to read/understand,
> and easy to get wrong.
> 
> Hence, move this data into separate ".dat" files and represent it in a way
> that can easily be read and rewritten by Perl scripts.  The new format is
> essentially "key => value" for each column; while it's a bit repetitive,
> explicit labeling of each value makes the data far more readable and less
> error-prone.  Provide a way to abbreviate entries by omitting field values
> that match a specified default value for their column.  This allows removal
> of a large amount of repetitive boilerplate and also lowers the barrier to
> adding new columns.
> 
> Also teach genbki.pl how to translate symbolic OID references into
> numeric OIDs for more cases than just "regproc"-like pg_proc references.
> It can now do that for regprocedure-like references (thus solving the
> problem that regproc is ambiguous for overloaded functions), operators,
> types, opfamilies, opclasses, and access methods.  Use this to turn
> nearly all OID cross-references in the initial data into symbolic form.
> This represents a very large step forward in readability and error
> resistance of the initial catalog data.  It should also reduce the
> difficulty of renumbering OID assignments in uncommitted patches.
> 
> Also, solve the longstanding problem that frontend code that would like to
> use OID macros and other information from the catalog headers often had
> difficulty with backend-only code in the headers.  To do this, arrange for
> all generated macros, plus such other declarations as we deem fit, to be
> placed in "derived" header files that are safe for frontend inclusion.
> (Once clients migrate to using these pg_*_d.h headers, it will be possible
> to get rid of the pg_*_fn.h headers, which only exist to quarantine code
> away from clients.  That is left for follow-on patches, however.)
> 
> The now-automatically-generated macros include the Anum_xxx and Natts_xxx
> constants that we used to have to update by hand when adding or removing
> catalog columns.
> 
> Replace the former manual method of generating OID macros for pg_type
> entries with an automatic method, ensuring that all built-in types have
> OID macros.  (But note that this patch does not change the way that
> OID macros for pg_proc entries are built and used.  It's not clear that
> making that match the other catalogs would be worth extra code churn.)
> 
> Add SGML documentation explaining what the new data format is and how to
> work with it.
> 
> Despite being a very large change in the catalog headers, there is no
> catversion bump here, because postgres.bki and related output files
> haven't changed at all.
> 
> John Naylor, based on ideas from various people; review and minor
> additional coding by me; previous review by Alvaro Herrera
> 
> Discussion: https://postgr.es/m/CAJVSVGWO48JbbwXkJz_yBFyGYW-M9YWxnPdxJBUosDC9ou_F0Q@mail.gmail.com
> 
> Branch
> ------
> master
> 
> Details
> -------
> https://git.postgresql.org/pg/commitdiff/372728b0d49552641f0ea83d9d2e08817de038fa
> 
> Modified Files
> --------------
> doc/src/sgml/bki.sgml                         |   661 +-
> doc/src/sgml/libpq.sgml                       |     2 +-
> src/Makefile                                  |     5 +
> src/backend/Makefile                          |    34 +-
> src/backend/catalog/.gitignore                |     2 +
> src/backend/catalog/Catalog.pm                |   313 +-
> src/backend/catalog/Makefile                  |    87 +-
> src/backend/catalog/README                    |   111 -
> src/backend/catalog/genbki.pl                 |   478 +-
> src/backend/utils/Gen_fmgrtab.pl              |    62 +-
> src/backend/utils/Makefile                    |    15 +-
> src/include/Makefile                          |     6 +-
> src/include/catalog/.gitignore                |     2 +
> src/include/catalog/Makefile                  |    31 +
> src/include/catalog/duplicate_oids            |     6 +-
> src/include/catalog/genbki.h                  |    29 +-
> src/include/catalog/indexing.h                |     2 +-
> src/include/catalog/pg_aggregate.dat          |   598 ++
> src/include/catalog/pg_aggregate.h            |   336 +-
> src/include/catalog/pg_am.dat                 |    34 +
> src/include/catalog/pg_am.h                   |    62 +-
> src/include/catalog/pg_amop.dat               |  2472 ++++++
> src/include/catalog/pg_amop.h                 |  1148 +--
> src/include/catalog/pg_amproc.dat             |  1277 +++
> src/include/catalog/pg_amproc.h               |   554 +-
> src/include/catalog/pg_attrdef.h              |    20 +-
> src/include/catalog/pg_attribute.h            |    59 +-
> src/include/catalog/pg_auth_members.h         |    22 +-
> src/include/catalog/pg_authid.dat             |    67 +
> src/include/catalog/pg_authid.h               |    71 +-
> src/include/catalog/pg_cast.dat               |   535 ++
> src/include/catalog/pg_cast.h                 |   368 +-
> src/include/catalog/pg_class.dat              |    68 +
> src/include/catalog/pg_class.h                |    76 +-
> src/include/catalog/pg_collation.dat          |    31 +
> src/include/catalog/pg_collation.h            |    42 +-
> src/include/catalog/pg_constraint.h           |    56 +-
> src/include/catalog/pg_conversion.h           |    29 +-
> src/include/catalog/pg_database.dat           |    26 +
> src/include/catalog/pg_database.h             |    34 +-
> src/include/catalog/pg_db_role_setting.h      |    29 +-
> src/include/catalog/pg_default_acl.h          |    27 +-
> src/include/catalog/pg_depend.h               |    49 +-
> src/include/catalog/pg_description.h          |    37 +-
> src/include/catalog/pg_enum.h                 |    27 +-
> src/include/catalog/pg_event_trigger.h        |    22 +-
> src/include/catalog/pg_extension.h            |    29 +-
> src/include/catalog/pg_foreign_data_wrapper.h |    23 +-
> src/include/catalog/pg_foreign_server.h       |    23 +-
> src/include/catalog/pg_foreign_table.h        |    19 +-
> src/include/catalog/pg_index.h                |    38 +-
> src/include/catalog/pg_inherits.h             |    24 +-
> src/include/catalog/pg_init_privs.h           |    42 +-
> src/include/catalog/pg_language.dat           |    31 +
> src/include/catalog/pg_language.h             |    39 +-
> src/include/catalog/pg_largeobject.h          |    19 +-
> src/include/catalog/pg_largeobject_metadata.h |    18 +-
> src/include/catalog/pg_namespace.dat          |    25 +
> src/include/catalog/pg_namespace.h            |    37 +-
> src/include/catalog/pg_opclass.dat            |   357 +
> src/include/catalog/pg_opclass.h              |   222 +-
> src/include/catalog/pg_operator.dat           |  3340 ++++++++
> src/include/catalog/pg_operator.h             |  1854 +----
> src/include/catalog/pg_opfamily.dat           |   248 +
> src/include/catalog/pg_opfamily.h             |   170 +-
> src/include/catalog/pg_partitioned_table.h    |    24 +-
> src/include/catalog/pg_pltemplate.dat         |    51 +
> src/include/catalog/pg_pltemplate.h           |    39 +-
> src/include/catalog/pg_policy.h               |    29 +-
> src/include/catalog/pg_proc.dat               | 10210 ++++++++++++++++++++++++
> src/include/catalog/pg_proc.h                 |  5637 +------------
> src/include/catalog/pg_publication.h          |    24 +-
> src/include/catalog/pg_publication_rel.h      |    19 +-
> src/include/catalog/pg_range.dat              |    31 +
> src/include/catalog/pg_range.h                |    61 +-
> src/include/catalog/pg_replication_origin.h   |    22 +-
> src/include/catalog/pg_rewrite.h              |    23 +-
> src/include/catalog/pg_seclabel.h             |    16 +-
> src/include/catalog/pg_sequence.h             |    20 +-
> src/include/catalog/pg_shdepend.h             |    42 +-
> src/include/catalog/pg_shdescription.h        |    36 +-
> src/include/catalog/pg_shseclabel.h           |    18 +-
> src/include/catalog/pg_statistic.h            |    44 +-
> src/include/catalog/pg_statistic_ext.h        |    26 +-
> src/include/catalog/pg_subscription.h         |    20 +-
> src/include/catalog/pg_subscription_rel.h     |    19 +-
> src/include/catalog/pg_tablespace.dat         |    22 +
> src/include/catalog/pg_tablespace.h           |    26 +-
> src/include/catalog/pg_transform.h            |    25 +-
> src/include/catalog/pg_trigger.h              |    35 +-
> src/include/catalog/pg_ts_config.dat          |    19 +
> src/include/catalog/pg_ts_config.h            |    29 +-
> src/include/catalog/pg_ts_config_map.dat      |    35 +
> src/include/catalog/pg_ts_config_map.h        |    47 +-
> src/include/catalog/pg_ts_dict.dat            |    20 +
> src/include/catalog/pg_ts_dict.h              |    31 +-
> src/include/catalog/pg_ts_parser.dat          |    20 +
> src/include/catalog/pg_ts_parser.h            |    58 +-
> src/include/catalog/pg_ts_template.dat        |    30 +
> src/include/catalog/pg_ts_template.h          |    49 +-
> src/include/catalog/pg_type.dat               |   942 +++
> src/include/catalog/pg_type.h                 |   565 +-
> src/include/catalog/pg_user_mapping.h         |    19 +-
> src/include/catalog/reformat_dat_file.pl      |   311 +
> src/include/catalog/toasting.h                |     2 +-
> src/include/catalog/unused_oids               |     8 +-
> src/interfaces/ecpg/ecpglib/pg_type.h         |    65 +-
> src/tools/msvc/Solution.pm                    |    49 +-
> src/tools/msvc/clean.bat                      |     2 +
> src/tools/pgindent/pgindent                   |     8 +-
> 110 files changed, 22928 insertions(+), 12480 deletions(-)
> 

-- 
Teodor Sigaev                      E-mail: teodor@sigaev.ru
                                       WWW: http://www.sigaev.ru/