goldengate同源一目標+多表和同源多目標+多表

ritchy發表於2018-10-23

小結一下,永記心中!
幾經修改,看見完美
曾經遇到的問題或值得注意的地方,就此記錄一下,以免再犯
開始。。。
******************同源一目標+多表******************
針對部分表進行OGG同步的話,可以參考下面的步驟進行操作,正常情況下按順序執行就差不多了
簡單操作的話可以按照下面試一下
【資料庫準備】
oracle環境略過
首先,源端開啟歸檔、全域性附加日誌和FORCELOGGING,目標端也開歸檔了,不知道不開行不行
檢查開啟歸檔
–檢視是否開啟
SQL> archive log list;
–開啟(須重啟DB)
SHUTDOWN IMMEDIATE;
STARTUP MOUNT;
ALTER DATABASE ARCHIVELOG;
ALTER DATABASE OPEN;
–檢查全域性附加日誌和FORCELOGGING
–查詢結果要是YES,不是的話執行ALTER,這個不需要重啟DB
SELECT SUPPLEMENTAL_LOG_DATA_MIN,FORCE_LOGGING FROM V$DATABASE;
ALTER DATABASE ADD SUPPLEMENTAL LOG DATA;
ALTER DATABASE FORCE LOGGING;

下面是源庫和目標庫都需要做的,可以同時做,下面都是在oracle使用者下執行的
0、環境變數設定
–下面是OGG的一些環境變數
export GG_HOME=/home/oracle/goldengate
export PATH=$GG_HOME:$PATH
–下面是OGG需要的動態連結庫,必須加上,否則ggsci就進不去
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$ORACLE_HOME/bin

1、建立ogguser使用者,是用來管理OGG的,與源端資料撇開關係
–建立的簡單,沒指定表空間也沒限額等等,只加了一些必要的授權
create user ogguser identified by ogguser;
grant connect to ogguser;
grant resource to ogguser;
grant unlimited tablespaces to ogguser;
grant execute on tul_file to ogguser;
grant select any dictionary to ogguser;
grant select any table to ogguser;
grant alter any table to ogguser;
grant flashback any table to ogguser;
grant execute on dbms_flashback to ogguser;

2、解壓安裝goldengate並建立目錄 ./ggsci、create subdirs
建立goldengate目錄,解壓檔案,goldengate這個也要給oracle許可權
mkdir -p /home/oracle/goldengate
tar -zxvf ggs_Adapters_Linux_x64_122.tar
執行報錯了可以參考下 https://www.cnblogs.com/ritchy/p/9851450.html
[oracle@rac1 goldengate]$ ./ggsci
GGSCI (rac1) 2> create subdirs

3、建立CKPT表
GGSCI (rac1) 1> edit param ./GLOBALS
ggschema ogguser
checkpointtable ogguser.checkpoint

GGSCI (DBDATA) 9> dblogin userid ogguser password ogguser
Successfully logged into database.

GGSCI (DBDATA as ogguser@test233) 10> add checkpointtable ogguser.checkpoint
Successfully created checkpoint table ogguser.checkpoint.

–如果出現已存在下面兩個表,通過sqlplus來drop,再重新執行上面的
sqlplus ogguser/ogguser
drop table checkpoint;
drop table checkpoint_lox;

4、建立並啟動MGR程式
GGSCI (rac1) 2> edit param mgr
PORT 7839
DYNAMICPORTLIST 7840-7914
AUTOSTART EXTRACT *
AUTORESTART EXTRACT *, RETRIES 3, WAITMINUTES 3
PURGEOLDEXTRACTS ./dirdat/*, USECHECKPOINTS, MINKEEPDAYS 5
LAGREPORTHOURS 1
LAGINFOMINUTES 30
LAGCRITICALMINUTES 45
–啟動
GGSCI (rac1 as ogguser@rac1) 15> start mgr
Manager started.

【源庫】
0、識別需要同步的表並新增附加日誌
[oracle@rac1 ~]$ sqlplus / as sysdba
ALTER TABLE TEST.T_ORDER ADD SUPPLEMENTAL LOG DATA (ALL) COLUMNS;
ALTER TABLE TEST.AA ADD SUPPLEMENTAL LOG DATA (ALL) COLUMNS;
ALTER TABLE TEST.BB ADD SUPPLEMENTAL LOG DATA (ALL) COLUMNS;

1、抽取程式 ext1
下面針對單例項 THREADS 1
ADD EXT ext1, TRANLOG, THREADS 1, BEGIN now
add exttrail ./dirdat/r1, extract ext1,megabytes 1000

GGSCI (rac1) 3> edit param ext1
EXTRACT ext1
setenv (NLS_LANG=AMERICAN_AMERICA.ZHS16GBK)
setenv (ORACLE_SID=rac1)
USERID ogguser, PASSWORD ogguser
REPORTCOUNT EVERY 1 MINUTES, RATE
NUMFILES 5000
discardfile ./dirrpt/ext.dsc,append, megabytes 100
DISCARDROLLOVER AT 3:00
EXTTRAIL ./dirdat/r1,megabytes 100
DYNAMICRESOLUTION
TRANLOGOPTIONS CONVERTUCS2CLOBS
TRANLOGOPTIONS DBLOGREADER
FETCHOPTIONS NOUSESNAPSHOT
FETCHOPTIONS FETCHPKUPDATECOLS
STATOPTIONS REPORTFETCH
WARNLONGTRANS 5h,CHECKINTERVAL 30m
—-TABLES——
table orders.t_order;
table orders.aa;
table orders.bb;

–啟動
start ext1

–資料庫環境變數字符集的查詢,上面的要與下面的對應
SQL> select * from nls_database_parameters;
PARAMETER VALUE
—————————— ————————————————–
–NLS_LANGUAGE AMERICAN
–NLS_TERRITORY AMERICA
NLS_CURRENCY $
NLS_ISO_CURRENCY AMERICA
NLS_NUMERIC_CHARACTERS .,
–NLS_CHARACTERSET ZHS16GBK

2、傳輸程式
add extract pxt1,exttrailsource ./dirdat/r1
add rmttrail ./dirdat/p1, extract pxt1,megabytes 1000
–其中 192.168.5.233 對應的是目標端的主機IP
GGSCI (rac1) 4> edit param pxt1
EXTRACT pxt1
DYNAMICRESOLUTION
PASSTHRU
RMTHOST 192.168.5.233, MGRPORT 7839, COMPRESS
RMTTRAIL ./dirdat/p1
NUMFILES 500
—tables
TABLE ORDERS.T_ORDER;
TABLE ORDERS.AA;
TABLE ORDERS.BB;

–啟動
start pxt1

3、注意事項:
*啟動這個之前要啟動目標機的MGR程式
*檢查源、目標庫抓取和獲取資料是否正常 ll dirdat
驗證源端抓取是否正常,已產生r1000000 檔案
[oracle@rac1 goldengate]$ ll dirdat
16
-rw-r—– 1 oracle oinstall 1370 10-17 18:33 r1000000

驗證目標端獲取資料是否正常,已產生 p1000000 檔案
[oracle@DBDATA goldengate]$ ll dirdat
total 12
-rw-r—– 1 oracle oinstall 0 Oct 17 18:46 p1000000

【目標庫資料初始化】
1、源庫獲取SCN
需要用到源庫SCN
col current_scn format 999999999999999
Select current_scn from v$database;
2、源+目標庫
create or replace directory dump_dir as `/home/oracle/dump_dir`;
grant read,write on directory dump_dir to ogguser;
3、EXPDP/IMPDP
源庫
expdp TEST/TEST directory=dump_dir tables=T_ORDER,aa,bb dumpfile=expdp_TEST_2tabs.dmp logfile=expdp_TEST_2tabs.log FLASHBACK_SCN=8689213127
–傳輸檔案到目標庫,然後就恢復
scp /mnt/dump_dir/expdp_TEST_2tabs* 192.168.5.233:/home/oracle/dump_dir
目標庫
impdp ogguser/ogguser directory=dump_dir remap_schema=TEST:ogguser dumpfile=expdp_TEST_bb.dmp logfile=expdp_TEST_bb_imp.log

【目標庫】
複製程式
用到的檔案/目錄是源端傳輸程式的p1,對應關係要一一對應
add replicat rxt1, exttrail ./dirdat/p1, checkpointtable ogguser.checkpoint

–mgr程式上面已經配置完成了
GGSCI (DBDATA) 20> edit param rxt1
replicat rxt1
handlecollisions
ASSUMETARGETDEFS
–setenv (ORACLE_SID=TEST233)
setenv (NLS_LANG=AMERICAN_AMERICA.ZHS16GBK)
userid ogguser,password ogguser
DBOPTIONS NOSUPPRESSTRIGGERS
REPORTCOUNT EVERY 1 MINUTES, RATE
–REPERROR DEFAULT, ABEND
NUMFILES 500
discardfile ./dirrpt/rxt.dsc, append, megabytes 100
ALLOWNOOPUPDATES
—-TABLES—–
map orders.t_order, target ogguser.t_order;
map orders.aa, target ogguser.aa;
map orders.bb, target ogguser.bb;

–啟動
start rxt1

注意事項
NOSUPPRESSTRIGGERS 目標端需要用到觸發器時,要加這個引數,否則觸發器不能用(OGG預設禁用的)。
至此OGG配置算差不多了。。。

【帶觸發器的表的同步測試】
–源表與目標表(結構一樣,初始從源庫impdp過來),必須有主鍵哦,主鍵是觸發器中的條件
–源端orders使用者下建立
create table BB
(
bb VARCHAR2(30) not null,
sdate DATE not null
);
alter table BB add constraint PK_B primary key (BB);

–目標端ogguser建立中間表,可不要主鍵
create table BB_CHANGES
(
bb VARCHAR2(30),
timestamp TIMESTAMP(6),
status NUMBER default 0
);

–目標端ogguser建立觸發器,源表主鍵必是其中的一個條件
CREATE OR REPLACE TRIGGER TR_BB_CHANGES
AFTER INSERT OR UPDATE ON BB
FOR EACH ROW
DECLARE
CNT NUMBER;
BEGIN
SELECT COUNT(1)
INTO CNT
FROM BB_CHANGES S
WHERE :NEW.BB = S.BB
AND S.STATUS = 0;
IF CNT = 0 THEN
INSERT INTO BB_CHANGES
(BB, TIMESTAMP)
VALUES
(:NEW.BB, CURRENT_TIMESTAMP);
END IF;
END;

在源庫orders下執行
INSERT INTO BB (BB, SDATE) VALUES (`1`, SYSDATE);
COMMIT;
INSERT INTO BB (BB, SDATE) VALUES (`2`, SYSDATE);
COMMIT;
INSERT INTO BB (BB, SDATE) VALUES (`3`, SYSDATE);
COMMIT;

UPDATE BB SET BB=`11` WHERE BB=`1`;
COMMIT;
–在目標庫可以看到BB表中已同步存在上面的資料,且BB_CHANGES中會增加相應的DML記錄
SELECT * FROM OGGUSER.BB ORDER BY SDATE DESC ;
SELECT * FROM OGGUSER.BB_CHANGES;

******************同源兩目標+多表******************
其實跟第一個差不多,如法炮製,可能比較笨,但可以互不干擾,便於維護
【源端】
1、抽取程式
ADD EXTRACT ext2, TRANLOG, THREADS 1, BEGIN now
add exttrail ./dirdat/r2, extract ext2,megabytes 100

GGSCI (rac1) 6> edit param ext2
EXTRACT ext2
setenv (NLS_LANG=AMERICAN_AMERICA.ZHS16GBK)
setenv (ORACLE_SID=rac1)
USERID ogguser, PASSWORD ogguser
REPORTCOUNT EVERY 1 MINUTES, RATE
NUMFILES 5000
discardfile ./dirrpt/ext.dsc,append, megabytes 100
DISCARDROLLOVER AT 3:00
EXTTRAIL ./dirdat/r2,megabytes 100
DYNAMICRESOLUTION
TRANLOGOPTIONS CONVERTUCS2CLOBS
TRANLOGOPTIONS DBLOGREADER
FETCHOPTIONS NOUSESNAPSHOT
FETCHOPTIONS FETCHPKUPDATECOLS
STATOPTIONS REPORTFETCH
WARNLONGTRANS 5h,CHECKINTERVAL 30m
—-TABLES——
table orders.t_order;
table orders.aa;
table orders.bb;

–啟動
start ext2

2、傳輸程式
add extract pxt2,exttrailsource ./dirdat/r2
add rmttrail ./dirdat/p2, extract pxt2,megabytes 100

–另外一個目標端是192.168.5.247
GGSCI (rac1) 7> edit param pxt2
EXTRACT pxt2
DYNAMICRESOLUTION
PASSTHRU
RMTHOST 192.168.5.247, MGRPORT 7839, COMPRESS
RMTTRAIL ./dirdat/p2
NUMFILES 5000
—tables
TABLE ORDERS.T_ORDER;
TABLE ORDERS.AA;
TABLE ORDERS.BB;

–啟動
start pxt2

【目標端】
其實與一個的一樣
mgr,imp初始化等略去。。。為了省事,也建立了同樣的ogguser,
複製程式來一發
add replicat rxt2, exttrail ./dirdat/p2, checkpointtable ogguser.checkpoint

GGSCI (localhost.localdomain as ogguser@test247) 53> edit param rxt2
replicat rxt2
handlecollisions
ASSUMETARGETDEFS
–setenv (ORACLE_SID=TEST247)
setenv (NLS_LANG=AMERICAN_AMERICA.AL32UTF8)
userid ogguser,password ogguser
DBOPTIONS NOSUPPRESSTRIGGERS
REPORTCOUNT EVERY 1 MINUTES, RATE
–REPERROR DEFAULT, ABEND
NUMFILES 500
discardfile ./dirrpt/rxt2.dsc, append, megabytes 100
ALLOWNOOPUPDATES
—-TABLES—–
map orders.t_order, target ogguser.t_order;
map orders.aa, target ogguser.aa;
map orders.bb, target ogguser.bb;

同上測試,同步正常
不會時候覺得無從下手,寢食難安,會懷疑自己的智商是不是回到幾千年了,多做幾次,通了,發現也就那關鍵的幾步,注意事項記在心就行了
當然生產可能規劃比實施更重要

相關文章