Thread: Issue import (COPY) data by between miss lines
Hi Admin, I have issue import data (COPY). Data content between miss lines Row 1: have 26 columns (DELIMITER BY ,) Row 2: have 24 column (DELIMITER BY ,) How to skip and write to log file this line ?? sample: Row1: 20150416ATM11429151780752,G6MOBILE,123PVTB,FARM,704450787,50000,10.30.81.2,704450787,MTgyIExlIERhaSBoYW5oIC0gUTExIC0gVFAgSENN,M,1/1/1970,0978233650,so6@vng.com.vn,Nạp ruby cho 704450787.,https://pay6.zing.vn/OrderResult.php?platform=&game_id=FARM&item_id=cnVieQ==,https://pay6.zing.vn/OrderResult.php?platform=&game_id=FARM&item_id=cnVieQ==,https://pay6.zing.vn/OrderResult.php?platform=&game_id=FARM&item_id=cnVieQ==,G6MOBILEkiMicJopM34kcam,bbac3d3f2615fcd1f1f4a9fe3a2c0c45bac009ab,,1,123P1504161455496,https://online.banknetvn.com.vn/pg/transprocess?Trans_id=c98d0f1693a49312b9011615066346a8,d3c516fb43a889ffa233674676188ff76e2957dd,Tao don hang thanh cong,2015-04-16 09:36:22 Row2: 20150416ATM21429151915479,G6MOBILE,123PBAB,FARM,50000,10.30.81.2,704443473,MTgyIExlIERhaSBoYW5oIC0gUTExIC0gVFAgSENN,M,01687302683,so6@vng.com.vn,Nạp ruby cho 704443473.,https://pay6.zing.vn/OrderResult.php?platform=&game_id=FARM&item_id=cnVieQ==,https://pay6.zing.vn/OrderResult.php?platform=&game_id=FARM&item_id=cnVieQ==,https://pay6.zing.vn/OrderResult.php?platform=&game_id=FARM&item_id=cnVieQ==,G6MOBILEkiMicJopM34kcam,0b4459ff26e11f4bf8bede11b4ee0129e42a2e25,,6203,,,,Khong ho tro ngan hang da chon,2015-04-16 09:38:35 20150416ATM21429151929774,G6MOBILE,123PVTB,FARM,704293984,50000,10.30.81.2,704293984,MTgyIExlIERhaSBoYW5oIC0gUTExIC0gVFAgSENN,M,1/1/1970,0972198329,so6@vng.com.vn,Nạp ruby cho 704293984.,https://pay6.zing.vn/OrderResult.php?platform=&game_id=FARM&item_id=cnVieQ==,https://pay6.zing.vn/OrderResult.php?platform=&game_id=FARM&item_id=cnVieQ==,https://pay6.zing.vn/OrderResult.php?platform=&game_id=FARM&item_id=cnVieQ==,G6MOBILEkiMicJopM34kcam,53e3082c97f26fc7632247d905471ac679ecf39b,,1,123P1504161455511,https://online.banknetvn.com.vn/pg/transprocess?Trans_id=0c23fe272e3fb15460f9153899648853,f6f7bff5dfc7f2d9bcd4bd2a98761e1911cf8530,Tao don hang thanh cong,2015-04-16 09:38:51 my code: COPY raw_table FROM '/data/scribe_log/CREATE_ORDER-2015-04-16_00009' WITH (FORMAT CSV, HEADER, DELIMITER E',', NULL '\n'); thanks, phuongnh2 -- View this message in context: http://postgresql.nabble.com/Issue-import-COPY-data-by-between-miss-lines-tp5848263.html Sent from the PostgreSQL - admin mailing list archive at Nabble.com.
I have retry this COPY raw_table FROM '/data/scribe_log/CREATE_ORDER-2015-04-16_00009' WITH (FORMAT CSV, HEADER, DELIMITER E',', NULL '\n',ERROR_LOGGING, ERROR_LOGGING_SKIP_BAD_ROWS); ERROR: option "error_logging" not recognized Current version: "PostgreSQL 9.4.1 on x86_64-unknown-linux-gnu, compiled by gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-11), 64-bit" How to fix it ?? thanks, phuongnh2 -- View this message in context: http://postgresql.nabble.com/Issue-import-COPY-data-by-between-miss-lines-tp5848263p5848264.html Sent from the PostgreSQL - admin mailing list archive at Nabble.com.
Hi Admin, How to config postgres to result below ?? CREATE TEMP TABLE foo (a bigint, b text); -- input_file.txt -- 1 one 2 3 three 111 four 4 5 five error logging off COPY foo FROM 'input_file.txt'; ERROR: missing data for column "b" CONTEXT: COPY foo, line 2: "2" skip bad rows --skip bad rows COPY foo FROM 'input_file.txt' (ERROR_LOGGING, ERROR_LOGGING_SKIP_BAD_ROWS); SELECT * FROM foo; a | b ---+------ 1 | one 5 | five (2 rows) -- View this message in context: http://postgresql.nabble.com/Issue-import-COPY-data-by-between-miss-lines-tp5848263p5848403.html Sent from the PostgreSQL - admin mailing list archive at Nabble.com.
Hi,
--
2015-05-08 6:41 GMT+02:00 phuongnh2 <phuongnh2@vng.com.vn>:
Hi Admin,
How to config postgres to result below ??
CREATE TEMP TABLE foo (a bigint, b text);
-- input_file.txt --
1 one
2
3 three 111
four 4
5 five
error logging off
COPY foo FROM 'input_file.txt';
ERROR: missing data for column "b"
CONTEXT: COPY foo, line 2: "2"
skip bad rows
--skip bad rows
COPY foo FROM 'input_file.txt' (ERROR_LOGGING,
ERROR_LOGGING_SKIP_BAD_ROWS);
SELECT * FROM foo;
a | b
---+------
1 | one
5 | five
(2 rows)
You can't use COPY do to that. That feature does not exist with PostgreSQL itself. pgloader should be able to do that, you should take a look.
--