[CentOS] Unable to grep 5 mins logs

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

[CentOS] Unable to grep 5 mins logs

ankush grover
Hi Friends!

I need to prepare a script which will grep logs from the current time
to previous 5 mins that is if the current time is Mon Jun 13 12:40:40
IST 2011 then all the logs between the interval Mon Jun 12:35 - 12:40
2011 should be grepped by the script and append it to another file.
However, the below script is not able to grep the desired logs, so I
need some help in preparing the script. I am running Centos 5.2
32-bit.



for (( i = 5; i >=0; i-- )) ; do grep $(date "+%a %b %d %R %Y" -d "-$i
 min") /var/ossec/logs/active-responses.log >> /tmp/newlog.log;done


/var/ossec/logs/active-responses.log format is below
Fri Jun  3 15:38:14 IST 2011
/var/ossec/active-response/bin/host-deny.sh add - 172.31.5.12
1307095694.71353 31151
Fri Jun  3 15:38:14 IST 2011
/var/ossec/active-response/bin/firewall-drop.sh add - 172.31.5.12
1307095694.71353 31151



Thanks & Regards


Ankush
_______________________________________________
CentOS mailing list
[hidden email]
http://lists.centos.org/mailman/listinfo/centos
Reply | Threaded
Open this post in threaded view
|

Re: [CentOS] Unable to grep 5 mins logs

John R Pierce
On 06/13/11 12:36 AM, ankush grover wrote:

> Hi Friends!
>
> I need to prepare a script which will grep logs from the current time
> to previous 5 mins that is if the current time is Mon Jun 13 12:40:40
> IST 2011 then all the logs between the interval Mon Jun 12:35 - 12:40
> 2011 should be grepped by the script and append it to another file.
> However, the below script is not able to grep the desired logs, so I
> need some help in preparing the script. I am running Centos 5.2
> 32-bit.
>
>
>
> for (( i = 5; i>=0; i-- )) ; do grep $(date "+%a %b %d %R %Y" -d "-$i
>   min") /var/ossec/logs/active-responses.log>>  /tmp/newlog.log;done
>
>
> /var/ossec/logs/active-responses.log format is below
> Fri Jun  3 15:38:14 IST 2011
> /var/ossec/active-response/bin/host-deny.sh add - 172.31.5.12
> 1307095694.71353 31151
> Fri Jun  3 15:38:14 IST 2011
> /var/ossec/active-response/bin/firewall-drop.sh add - 172.31.5.12
> 1307095694.71353 31151

Well,

$ i=5 date "+%a %b %d %R %Y" -d "-$i min"
Mon Jun 13 00:46 2011

so that probably won't work for matching the text in your logfiles...


--
john r pierce                            N 37, W 122
santa cruz ca                         mid-left coast

_______________________________________________
CentOS mailing list
[hidden email]
http://lists.centos.org/mailman/listinfo/centos
Reply | Threaded
Open this post in threaded view
|

Re: [CentOS] Unable to grep 5 mins logs

Ljubomir Ljubojevic
John R Pierce wrote:

> On 06/13/11 12:36 AM, ankush grover wrote:
>> Hi Friends!
>>
>> I need to prepare a script which will grep logs from the current time
>> to previous 5 mins that is if the current time is Mon Jun 13 12:40:40
>> IST 2011 then all the logs between the interval Mon Jun 12:35 - 12:40
>> 2011 should be grepped by the script and append it to another file.
>> However, the below script is not able to grep the desired logs, so I
>> need some help in preparing the script. I am running Centos 5.2
>> 32-bit.
>>
>>
>>
>> for (( i = 5; i>=0; i-- )) ; do grep $(date "+%a %b %d %R %Y" -d "-$i
>>   min") /var/ossec/logs/active-responses.log>>  /tmp/newlog.log;done
>>
>>
>> /var/ossec/logs/active-responses.log format is below
>> Fri Jun  3 15:38:14 IST 2011
>> /var/ossec/active-response/bin/host-deny.sh add - 172.31.5.12
>> 1307095694.71353 31151
>> Fri Jun  3 15:38:14 IST 2011
>> /var/ossec/active-response/bin/firewall-drop.sh add - 172.31.5.12
>> 1307095694.71353 31151
>
> Well,
>
> $ i=5 date "+%a %b %d %R %Y" -d "-$i min"
> Mon Jun 13 00:46 2011
>
> so that probably won't work for matching the text in your logfiles...
>
>
Combine 2-3 greps:

for (( i = 5; i>=0; i-- )) ; do grep `date "+%a"` | grep `date "+%b"` |
grep `date "+%d"` | grep `date "+%Y"` | $(date "+%R" -d "-$i
min") /var/ossec/logs/active-responses.log>>  /tmp/newlog.log;done

Change order of greps to gain speed at first cutting part of lines with
most hits.

Ljubomir
_______________________________________________
CentOS mailing list
[hidden email]
http://lists.centos.org/mailman/listinfo/centos
Reply | Threaded
Open this post in threaded view
|

Re: [CentOS] Unable to grep 5 mins logs

ankush grover
Combine 2-3 greps:

>
> for (( i = 5; i>=0; i-- )) ; do grep `date "+%a"` | grep `date "+%b"` |
> grep `date "+%d"` | grep `date "+%Y"` | $(date "+%R" -d "-$i
> min") /var/ossec/logs/active-responses.log>>  /tmp/newlog.log;done
>
> Change order of greps to gain speed at first cutting part of lines with
> most hits.
>
> Ljubomir
> _

It is really slow when 2-3 greps are combined.

______________________________________________
> CentOS mailing list
> [hidden email]
> http://lists.centos.org/mailman/listinfo/centos
>
_______________________________________________
CentOS mailing list
[hidden email]
http://lists.centos.org/mailman/listinfo/centos
Reply | Threaded
Open this post in threaded view
|

Re: [CentOS] Unable to grep 5 mins logs

Ljubomir Ljubojevic
ankush grover wrote:

> Combine 2-3 greps:
>> for (( i = 5; i>=0; i-- )) ; do grep `date "+%a"` | grep `date "+%b"` |
>> grep `date "+%d"` | grep `date "+%Y"` | $(date "+%R" -d "-$i
>> min") /var/ossec/logs/active-responses.log>>  /tmp/newlog.log;done
>>
>> Change order of greps to gain speed at first cutting part of lines with
>> most hits.
>>
>> Ljubomir
>> _
>
> It is really slow when 2-3 greps are combined.
>

But it will do the job until you solve this with more elegance.

What you can try is to compile search pattern from 2-3 date outputs so
it will match the text in the log.


dayname="$(date "+%a")"; month="$(date "+%b")"; time="$(date "+%d")";
year="$(date "+%Y")";
search1="$dayname  $month $time $year" # add spaces where needed and
order parts properly to match log
for (( i = 5; i>=0; i-- )) ; do grep $(date "+%R" -d "-$i
min") /var/ossec/logs/active-responses.log | grep $search1 >>
/tmp/newlog.log;done

Also consider dropping parts like day as a name when you have day as a
number to speed up.

Ljubomir
_______________________________________________
CentOS mailing list
[hidden email]
http://lists.centos.org/mailman/listinfo/centos
Reply | Threaded
Open this post in threaded view
|

Re: [CentOS] Unable to grep 5 mins logs

ankush grover
>>
>> It is really slow when 2-3 greps are combined.
>>
>
> But it will do the job until you solve this with more elegance.
>
> What you can try is to compile search pattern from 2-3 date outputs so
> it will match the text in the log.
>
>
> dayname="$(date "+%a")"; month="$(date "+%b")"; time="$(date "+%d")";
> year="$(date "+%Y")";
> search1="$dayname  $month $time $year" # add spaces where needed and
> order parts properly to match log
> for (( i = 5; i>=0; i-- )) ; do grep $(date "+%R" -d "-$i
> min") /var/ossec/logs/active-responses.log | grep $search1 >>
> /tmp/newlog.log;done
>
> Also consider dropping parts like day as a name when you have day as a
> number to speed up.
>
> Ljubomir

Thanks a lot Ljubomir :)

The script is below

month="$(date "+%b")"; time="$(date "+%d")";year="$(date "+%Y")";
search1="$month  $time"
echo "$search1"
for (( i = 5; i>=0; i-- )) ; do grep $(date "+%R" -d "-$i min")
/var/ossec/logs/active-responses.log | grep "$search1" | grep "$year"
>> /tmp/ossecactive.log;done
_______________________________________________
CentOS mailing list
[hidden email]
http://lists.centos.org/mailman/listinfo/centos