Filter by datetime (fd)

The fd tool is a program designed to filter specially log files (byte encoded) by date and time, up to a precision of seconds. It takes one or more log files as input and filters the entries based on a specified start and end date and time. The tool’s debugging feature can help troubleshoot potential issues. The tool can replace the datetime format in the output with the ‘YYYY-MM-DD hh:mm:ss’ format. This helps in standardising the datetime format across different log files. It’s useful to analyse log files within a specific timeframe.

fd
Start- or End- Date must be given

fd --help
F. P.:  Program to filter carmen/yoda logfiles(byte encoded) by datetime. Only till seconds (no millis and more)
 
Usage: fd [OPTIONS] [FILES]...
 
Arguments:
  [FILES]...  Name of the files to filter
 
Options:
  -d, --debug...       Turn debugging information on 
  -r, --replace        replace input with YYYY-MM-DD hh:mm:ss(2023-01-24 13:57:31) in output
  -s, --start <START>  The optional start date: dd.mm.yyyy HH::MM:SS
  -e, --end <END>      The optional end date: dd.mm.yyyy HH::MM:SS
  -h, --help           Print help
  -V, --version        Print version

Examples:

TWP_err_4.log:  
...
20240502124631;HOST;;TPW;8484;E;0;0 Nr: 9999 Message: ssl_accept(...) wurde fuer 1.2.3.4:49352 auf fd=46 mit Timeout(3 s) beendet. 0  /users/.../.../FPE_ACE_SSL_SOCK_Acceptor.cpp 191 FPE_ACE_SSL_SOCK_Acceptor 0;
20240502125955;HOST;;TPW;8484;E;0;0 Nr: 9999 Message: ssl_accept(...) wurde fuer 1.2.3.4:59412 auf fd=46 mit Timeout(3 s) beendet. 0  /users/.../.../FPE_ACE_SSL_SOCK_Acceptor.cpp 191 FPE_ACE_SSL_SOCK_Acceptor 0;
...

fd -r -s"2.5.24 12:0:0" -e"2.5.24 13:0:0" TPW_err_4.log

2024-05-02 12:46:31;HOST;;TPW;8484;E;0;0 Nr: 9999 Message: ssl_accept(...) wurde fuer 1.2.3.4:49352 auf fd=46 mit Timeout(3 s) beendet. 0  /users/.../.../FPE_ACE_SSL_SOCK_Acceptor.cpp 191 FPE_ACE_SSL_SOCK_Acceptor 0;
2024-05-02 12:59:55;HOST;;TPW;8484;E;0;0 Nr: 9999 Message: ssl_accept(...) wurde fuer 1.2.3.4:59412 auf fd=46 mit Timeout(3 s) beendet. 0  /users/.../.../FPE_ACE_SSL_SOCK_Acceptor.cpp 191 FPE_ACE_SSL_SOCK_Acceptor 0; 

fd -r -s"7.5.24 11:0:0" -e"7.5.24 11:01:0" server.log

2024-05-07 11:00:00,955 SEVERE [de.X.crm.core.config.DuckHawkConfigurationManager] (DuckHawkConfigurationManager reloading configfiles threadID: 609) loadPropertyFile: ../standalone/configuration/duckhawk.properties for config interface de.X.duckhawk.webservicebase.housekeeping.HouseKeepingLogfiles$Config failed! Path=/.../standalone/configuration/duckhawk.properties
2024-05-07 11:00:00,955 SEVERE [de.X.crm.core.config.DuckHawkConfigurationManager] (DuckHawkConfigurationManager reloading configfiles threadID: 609) loadPropertyFile: ../standalone/configuration/duckhawk.properties for config interface de.X.crm.core.config.DuckHawkConfig failed! Path=/.../standalone/configuration/duckhawk.properties
2024-05-07 11:00:06,280 SEVERE [net.X.crm.lifecycle.schedule.LifeCycleScheduler] (pool-17-thread-1) Break [exportXml] loop [LCM-XML-Exporter: Currently NO notification available in database!]
2024-05-07 11:00:06,283 INFO  [net.X.crm.lifecycle.schedule.LifeCycleScheduler] (pool-17-thread-1) Starting delay before next loop of [30000] (ms).
2024-05-07 11:00:36,286 SEVERE [net.X.crm.lifecycle.schedule.LifeCycleScheduler] (pool-17-thread-1) Break [exportXml] loop [LCM-XML-Exporter: Currently NO notification available in database!]
2024-05-07 11:00:36,288 INFO  [net.X.crm.lifecycle.schedule.LifeCycleScheduler] (pool-17-thread-1) Starting delay before next loop of [30000] (ms).

fd -r -s"7.5.24 11:0:28" -e"7.5.24 11:00:28" SA_prot_2.log SA_deb_2.log SA_err_2.log

2024-05-07 11:00:28 M     0 FILE /.../.../CRequestToolBoxRemote.cpp:319 [CRequestRemoteModel] PID: 1070 ServiceCall CPRM::BusinessParty::InvoiceReadServices.getInvoiceInformation5 CorrelationId: XYZXYZXYZXYZXYZXYZXYZXYZXYZXYZ
2024-05-07 11:00:28 M     0 FILE /.../.../CCMOwnerIDManager.cpp:72 [CCMOwnerIDManager] PID: 1070 OwnerId: 1 in Basisschicht gesetzt.
2024-05-07 11:00:28 M     0 FILE /.../.../pointofcontact/S_Address2.cpp:47 [CPRM::PointOfContact::collectS_Address2] PID: 1070 Overflow ReachabilityId <217xxxxxxxx> setting [s_Address2.handle=-1] to indicate.
2024-05-07 11:00:28 M     0 FILE /.../.../CCMOwnerIDManager.cpp:72 [CCMOwnerIDManager] PID: 1070 OwnerId: 1 in Basisschicht gesetzt.
2024-05-07 11:00:28 M     0 FILE /.../.../CCTimeLog.cpp:65 [CC-Service] PID: 1070 CPRM::BusinessParty::InvoiceReadServices.getInvoiceInformation5.0075040604:.XYZXYZXYZXYZXYZXYZXYZXYZXYZXYZ.runtime : user 0,006543 sys 0,000000 sum 0,006543 sec real 57 msec

Convert to ‘YYYY-MM-DD hh:mm:ss’ format

cat TWC_err_2.log | fd -r -s"1.1.1 0:0:0" > TWC_err_2.log.std