Would you be so kind to help me with some regex and Python code? I have recently received a lot of help for a Python script to go through some firewall logs. I am almost there, but still need a little help.
Here some log output from a firewall:
Nov 11 00:00:09 firewall %ASA-3-710003: TCP access...
The reason why I've created this room, is that it seems like you've misunderstood the timing an sequencing of for loops related to reading files. So I would like to help you understand that in particualr, instead of just posting the solution to your code.
Before going to that, know that if you post a message with @holroy in this room, I'll get ping'ed, even if I'm not presently in the room. So please do leave comments, question and answers, even though I'm not currently around. If I'm pingen I'll reenter within a day or two, and hopefully be able to respond. :-)
Given a text file like the following:
Nov 11 00:00:09 firewall %ASA-3-710003: TCP access denied by ACL from 1.1.1.1/50624 to internet:2.2.2.2/80
Nov 6 12:42:23 firewall %ASA-4-106023: Deny tcp src inside:3.3.3.3/42059 dst internet:4.4.4.4/389 by access-group "access_out" [0x0, 0x0]
And the following code snippet:
for line in data : #a
if 'Deny ' in line : #b
seen.update(DENY_PATTERN.findall(line)) #c
for line in data : #d
if 'ASA-3 ' in line : #e
seen.update(DENY_PATTERN2.findall(line)) #f
Could you please try to explain, textually, when the different lines of code is executed for which line in the text file? This sounds kind of basic, but I'm wondering if you misunderstood a vital piece here.
In regards to the code snippet, I was trying to go through the lines in the data file and search for Deny statement. If found it should "execute" the regex specified by DENY_PATTERN. The second loop does the same for the ASA-3. If found it executes the regex specified in DENY_PATTERN2
I did at some point try to make just one regex with a or statement in between like so: shared regex, specific regex with or in between, shared regex, but didn't get i to work. I also tried with just one loop where both if statements where specified, but also didn't get it to work.
But I actually think I didn't get i to work wasn't the loops, nor the regex but maybe this part: for (protocol, src, dst, _, _, dst_port), count in counter.most_common(): where I got some errors in regards to too few/may values.
In regards to the regex understanding (how do you make CR in chat :o) ), well [A-Z].* matches all uppercase characters between A-Z and the .* part matches everything forward until it is "stopped" somehow (greedy). [A-Z]* matches all uppercase chars between A-Z until next whitespace.
The reason I asked, was that if you have two of the for line in data, you don't have anything left of the file for the second run. You need to include both if tests within the first loop.
@joni In your case, the second loop would never find anything as you don't have more lines in the file...
@joni Building a comples regex for both these cases would make for a very hard to read regexp. It could be done using (?:<first regexp>|<second regexp). Note also the use of (?: ... ) to make a non-capturing group, which eliminate some of the matches.
@joni Getting the print output to match when you have two different regexp pattenrs with different output, requires a little rewrite where you in the for loop catches the full tuple of the match, and then within the for loop you differentiate either on the length of the tuple match, or one a possibly separate inclusion of an identifier. In your case you could include Deny or ASA-3 if the number of captured groups where equal
Or a slightly more hackish solution could be to include an empty capturing group in one of the regexp's to change the numbers... ;-/
@joni I don't really like the construct [A-Z].*?, as it matches only one uppercase character, and then a lot of other stuff. This would match against TCPoodle,dofkejroelj/120283 just nicely, whilst you really want to match on a word of uppercase characters, which is [A-Z].*?, which in previous match would only give you TCP
On a slightly related note, when using regex101.com, do remember to tick off the python flavor in the left side menu. That would remove the need for \/, and changes some other stuff... I took the liberty of updating your regexp, and ended up with regex101.com/r/wC6gS7/8
Testing multiple patterns
To test for multiple complex pattern, you'll want to add those tests within the one and only loop through your data file. That is you want the following code:
for line in data :
# If line has 'Deny ' in it, then check it some more
if 'Deny ' in lin...
Would you be so kind to help me with some regex and Python code? I have recently received a lot of help for a Python script to go through some firewall logs. I am almost there, but still need a little help.
Here some log output from a firewall:
Nov 11 00:00:09 firewall %ASA-3-710003: TCP access...
Testing multiple patterns
To test for multiple complex pattern, you'll want to add those tests within the one and only loop through your data file. That is you want the following code:
for line in data :
# If line has 'Deny ' in it, then check it some more
if 'Deny ' in lin...
A Python room for me and Joni
A Python room for me and Joni