Versions Compared

Version Old Version 12 New Version 13
Changes made by

Former user

Former user

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Collector manager를 마크베이스 서버와 연결하기 위해서 마크베이스 서버에 collector manager를 등록한다. 아래의 명령을 machsql을 이용하여 실행한다.

Syntax문법:

Code Block
languagesql
CREATE COLLECTORMANAGER manager_name AT "host_addr:host_port";

...

Collector manager를 등록한 이후, collector객체를 collector manager를 통해서 생성한다. Collector에 대한 정보는 마크베이스 서버에 저장되고, 그 정보를 조회할 수 있다. 아래의 명령을 machsql을 통해서 실행하여 collector를 생성한다.

syntax문법:

Code Block
CREATE COLLECTOR manager_name.collector_name FROM "path_for_template.tpl";

...

Name of variableDescriptionDetailed information
NAMEColumn name of tableIt needs character strings without a space.
TYPEData type of a tableIt represents the type of a string. Types and strings are as follows. The size is in the parentheses. (short (6), int (11), long (20), float (17), double (17), datetime (31), varchar (User-defined), ipv4 (15), ipv6 (45), text (64MB), binary (64MB))
SIZEIt indicates the size of the column.The size has to be the same with the specified size listed above except varchar type.
DATE_FORMATWhen TYPE is datetime, format string to express the time. e.g.) Aug 19 07:56:16 is the format of month day hour:min:sec, and it is required to define "%b %d %H:%M:%S" in order to parse them with "strptime".It uses the internal system function "strptime" when parsing. And it has to describe month, day, hour or year in the exact format that the system function supports.
USE_INDEXIndex creation flag for the column: 0: Do not create/ 1: CreateIt is automatically generated based on data types.
REGEX_NONumber of result values from the regular expression that is entered as REGEX.When the input log string is parsed, it is printed in multiple strings. In this case, it needs to determine what to use for the current column from the several results and it records the number of the result as well. When it's parsed, the utility machregex can be used to see the string.

...

syslog.tpl 예제

아래에 syslog.tpl파일의 예제를 기술한다. 해당 파일은 $MACHBASE_COLLECTOR_HOME/collector/syslog.tpl에 샘플로 제공된다.

...

Code Block
###############################################################################
# Copyright of this product 2013-2023,
# Machbase Corporation (Incorporation) or its subsidiaries.
# All Rights reserved
###############################################################################

#
#  This file is for Machbase collector regex file.
#

LOG_TYPE=syslog

COL_LIST= (
     (
        REGEX_NO = 0 <== It is the number included in the results of machregex.
        NAME = tm
        TYPE = datetime
        SIZE = 8
        DATE_FORMAT="%b %d %H:%M:%S" <== Use dateformat of strptime.
         ),
     (
        REGEX_NO = 4
        NAME = host
        TYPE = varchar
        SIZE = 128
USE_INDEX = 1 <== Determine whether to use the index.
         ),
     (
        REGEX_NO = 5
        NAME = msg
        TYPE = varchar
        SIZE = 512
USE_INDEX = 1
         )
)

# Below is the regular expression to pares syslog data. It may not work properly if it is modified.
REGEX="(([a-zA-Z]+)\s+([0-9]+)\s+([0-9:]*))\s(\S+)\s+([^\n]+)"

END_REGEX="\n"

...


Collector 생성

아래와 같이 collector "syslog_test"를 생성한다.

Code Block
languagesql
mach> CREATE COLLECTOR localhost.syslog_test FROM "/home/mach/mach_collector_home/collector/syslog.tpl";
Created successfully.

...


Collector 확인

M$SYS_COLLECTORS 테이블에 등록된 collector들의 정보가 있다. "RUN_FLAG" 컬럼 값이 1인 collector는 실행중이며 0이면 실행이 중지된 상태이다.

...

Code Block
mach> SELECT collector_name, run_flag FROM m$sys_collectors;
collector_name                            run_flag    
---------------------------------------------------------
SYSLOG_TEST                               0           
[1] row(s) selected.
mach> SELECT * FROM m$sys_collectors;
COLLECTOR_ID MANAGER_NAME                                 COLLECTOR_NAME                            
-----------------------------------------------------------------------------------------------------
LOG_TYPE                                  TABLE_NAME                                
---------------------------------------------------------------------------------------
TEMPLATE_NAME                                                                     COLLECT_TYPE                              
-------------------------------------------------------------------------------------------------------------------------------
COLLECTOR_SOURCE                                                                  
------------------------------------------------------------------------------------
COLLECTOR_LIB                                                                     COL_COUNT   
-------------------------------------------------------------------------------------------------
PREPROCESS_PATH                                                                   
------------------------------------------------------------------------------------
REGEX_PATH                                                                        
------------------------------------------------------------------------------------
REGEX                                                                             
------------------------------------------------------------------------------------
END_REGEX                                                                         
------------------------------------------------------------------------------------
DEFAULT_ADDR                                                                      LANGUAGE                          
-----------------------------------------------------------------------------------------------------------------------
SLEEP_TIME  DB_ADDR                                   DB_PORT     DB_USER                                   
-----------------------------------------------------------------------------------------------------------------
DB_PASS                                   RUN_FLAG    
---------------------------------------------------------
1           LOCALHOST                                 SYSLOG_TEST                               
syslog                                    syslogtable                               
/home/mach/mach_collector_home/collector/syslog.tpl                             FILE                                      
/var/log/syslog                                                                   
NULL                                                                              7           
NULL                                                                              
syslog.rgx                                                                        
(([a-zA-Z]+)\s+([0-9]+)\s+([0-9:]*))\s(\S+)\s+([^\n]+)                            
\n                                                                                
192.168.122.1                                                                     UTF-8                             
1000        127.0.0.1                                 5656        SYS                                       
MANAGER                                   0           
[1] row(s) selected.

...


Collector 실행

Syntax문법:

Code Block
languagesql
ALTER COLLECTOR manager_name.collector_name START [TRACE];

...

Collector 실행시 오류가 발생한 경우 $MACHBASE_COLLECTOR_HOME/trc/machcollector.trc 파일을 참고하여 문제 해결을 할 수 있다.

Example예제:

Code Block
mach> ALTER COLLECTOR localhost.syslog_test START;
Altered successfully.
mach> SELECT collector_name, run_flag FROM m$sys_collectors;
collector_name                            run_flag    
---------------------------------------------------------
SYSLOG_TEST                               1           
[1] row(s) selected.

ALTER COLLECTOR문으로 collector를 시작하면 RUN_FLAG 컬럼 값이 1 변경된 것을 알 수 있다. Collector를 시작하면 수집된 데이터가 저장되는 log 테이블도 데이터베이스 서버에 생성된다. collector_type, collector_addr, collector_origin, collector_offset 값들은 기본 값으로 설정된다. syslog.tpl 파일에 설정된 tmp, host, msg 컬럼 또한 생성된다.

Example예제:

Code Block
languagesql
mach> ALTER COLLECTOR localhost.syslog_test START;
Altered successfully.
mach> SELECT collector_name, run_flag FROM m$sys_collectors;
collector_name                            run_flag    
---------------------------------------------------------
SYSLOG_TEST                               1           
[1] row(s) selected.

...

Collector 실행시 오류가 발생하여 중지된 이후, 재 실행되면 콜렉터는 마지막으로 입력한 데이터의 위치를 읽어 들여서 데이터를 유실하지 않고 나머지 데이터를 계속 입력한다.

...

데이터 확인

아래 내용은 원본 syslog의 마지막 10건과 데이터와 입력된 데이터를 비교한 것이다.

...

Code Block
mach> SELECT collector_name, run_flag FROM m$sys_collectors;
collector_name                            run_flag    
---------------------------------------------------------
SYSLOG_TEST                               1           
[1] row(s) selected.

...

Collector 정지

Syntax문법:

Code Block
ALTER COLLECTOR manager_name.collector_name STOP;


Example예제:

Code Block
mach> ALTER COLLECTOR localhost.syslog_test STOP;
Altered successfully.

...

Code Block
mach> ALTER COLLECTOR localhost.syslog_test STOP;
Altered successfully.

...


Collector 삭제

Syntax문법:

Code Block
DROP COLLECTOR manager_name.collector_name;


Example예제:

Code Block
mach> DROP COLLECTOR localhost.syslog_test;
Dropped successfully.

...

Code Block
mach> SELECT collector_name, run_flag FROM m$sys_collectors;
collector_name                            run_flag    
---------------------------------------------------------
[0] row(s) selected.

Collector

...

갱신

Syntax문법:

Code Block
ALTER COLLECTOR manager_name.collector_name RELOAD;

Example예제:

콜렉터를 생성한 이후, 템플릿 파일을 변경하고 그 내용을 새로 적용할 경우에 사용한다. 실행시 갱신된 템플릿 파일의 내용이 적용된다. 아래 예제는 데이터를 입력할 테이블을 원래 값 대신에 "anothertable"로 바꾼 경우이다.

...