pgbigm / pg_bigm Goto Github PK

Hi there,

It seems #10 is already fixed by 34fe30f and it's already back-ported to REL1_2_STABLE branch.

Would it make sense to tag the current REL1_2_STABLE branch and create a new release with it?
I'm aware that the full migration to GitHub is still being planned, but making a new release at least provides people a stable package location with Postgres 16 support.

https://osdn.net/projects/pgbigm/releases/72448

The download is constantly falling off by timeout:
https://osdn.dl.osdn.net/pgbigm/72448/pg_bigm-llvmjit-1.2.20200228-1.pg15.el9.x86_64.rpm

especially big problems started last week

In my environment, I ran pg_bigm's regression tests with memory sanitization checks turned on and got a warning about a use of uninitialized memory below:

==2630==WARNING: MemorySanitizer: use-of-uninitialized-value
#0 0x5595cb3cdc43 in ginFillScanEntry postgres/src/backend/access/gin/ginscan.c:72:6
#1 0x5595cb3cdc43 in ginFillScanKey postgres/src/backend/access/gin/ginscan.c:221:23
#2 0x5595cb3cd045 in ginNewScanKey postgres/src/backend/access/gin/ginscan.c:372:3
#3 0x5595cb3ce63d in gingetbitmap postgres/src/backend/access/gin/ginget.c:1827:2
#4 0x5595cb2024ef in index_getbitmap postgres/src/backend/access/index/indexam.c:600:10
#5 0x5595cb309512 in MultiExecBitmapIndexScan postgres/src/backend/executor/nodeBitmapIndexscan.c:89:23
#6 0x5595cb2c6b5d in BitmapHeapNext postgres/src/backend/executor/nodeBitmapHeapscan.c:104:23
#7 0x5595cb2c8687 in ExecScan postgres/src/backend/executor/execScan.c:180:10
#8 0x5595cb2b8e84 in ExecProcNode postgres/src/backend/executor/execProcnode.c:437:13
#9 0x5595cb205914 in ExecutePlan postgres/src/backend/executor/execMain.c:1567:10
#10 0x5595cb205914 in standard_ExecutorRun postgres/src/backend/executor/execMain.c:339:3
#11 0x5595cb66bffe in PortalRunSelect postgres/src/backend/tcop/pquery.c:948:4
#12 0x5595cb66b556 in PortalRun postgres/src/backend/tcop/pquery.c:789:18
#13 0x5595cb664bbe in exec_simple_query postgres/src/backend/tcop/postgres.c:1114:10
#14 0x5595cb664bbe in PostgresMain postgres/src/backend/tcop/postgres.c:4185:7
#15 0x5595cb56d7b9 in BackendRun postgres/src/backend/postmaster/postmaster.c:4430:2
#16 0x5595cb56c094 in BackendStartup postgres/src/backend/postmaster/postmaster.c:4096:3
#17 0x5595cb56c094 in ServerLoop postgres/src/backend/postmaster/postmaster.c:1759:7
#18 0x5595cb5669e4 in PostmasterMain postgres/src/backend/postmaster/postmaster.c:1367:11
#19 0x5595cb1662ca in main postgres/src/backend/main/main.c:228:3

LOG: server process (PID 2630) exited with exit code 66
DETAIL: Failed process was running: SELECT col1 FROM test_bigm WHERE col1 LIKE likequery('%');
LOG: terminating any other active server processes
WARNING: terminating connection because of crash of another server process
DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.

I looked into it and recognized that pg_bigm allocates a bool and stores it extra_data in gin_extract_query_bigm. However, Postgres' GIN index expects the extractQuery implementation to store an array of *nkeys pointers (doc) in extra_data. As Postgres tries to copy the array pointed to by extra_data, it results in the warning.

Do we think it is a real issue and if and how it can be resolved? Thanks.

I am trying to open http://osdn.jp/projects/pgbigm/releases/?package_id=13634. The site sometimes opens but takes like 1 minute to load.

Hi there, so postgres 16 beta 1 is out and beta 2 has been stamped, may I know when we can expect a pg 16 compatible version to be available? Thanks!

writer: r5.large
standby: r6g.large

step1: writer

CREATE EXTENSION pg_bigm;
CREATE INDEX ix_t1_content ON t1 using GIN (content gin_bigm_ops);

insert into t1 (col1, col2, content)
SELECT
gs10, gs20,
concat(md5(random()::text)) AS random_string
FROM
generate_series(1, 1) AS gs;

update t1 set content = concat(content,'가나다') where id = 1;

rdsadmin=# select * from t1 where content like '%가나다%';
id | col1 | col2 | content
----+------+------+----------------------------------------
1 | 10 | 20 | 7329992bb1dc5fd43140f6c596cd035c가나다
(1 row)

step2: standby reader

rdsadmin=# select * from t1 where content like '%가나다%';
id | col1 | col2 | content
----+------+------+---------
(0 rows)

rdsadmin=# select * from t1 where content like '%bb1dc5fd431%';
id | col1 | col2 | content
----+------+------+----------------------------------------
1 | 10 | 20 | 7329992bb1dc5fd43140f6c596cd035c가나다
(1 row)

Due to issues with OSDN, such as their ML service being currently unavailable, I propose a full migration of the pg_bigm official site from OSDN to GitHub step by step. To achieve this, at first I suggest the following steps:

1. Rename the current html/ directory to docs/ to enable easy reading of all pg_bigm docs (html files) on GitHub, rather than OSDN.
2. Update README.md so that it points those docs on GitHub.
3. Create all releases that have already been created in OSDN, on GitHub, and also upload all assets, such as RPM files, to them.

This discussion basically should be taken place on the pgbigm-hackers ML. However, since the ML service is currently out of order, I have posted this proposal as an issue on GitHub. Any comments or feedback would be greatly appreciated.

Hi.
Woud you show me the .spec file for rpmbuild ?
(detail is https://ja.osdn.net/projects/pgbigm/forums/31032/44981/ )

Note that bigm_similarity is NOT case-sensitive, but pg_trgm's similarity function is case-sensitive. should be Note that bigm_similarity IS case-sensitive, but pg_trgm is not.
https://github.com/pgbigm/pg_bigm/blob/master/html/pg_bigm_en-1-2.html#L379

Cheers!