Member Avatar for zyrus001

Hi,

I'm pretty new to python but thought I'd write something to read to try and learn it. Ideally, it will read this log file, match a string and return everything between the "****" delimiters where the string was matched. I'm not 100% sure how to go about this though. I populate my tuple with the following, but can't see how I display the lines inbetween the delimiter objects.

def look(self):
        
        NULL = None
        data = [(0,0,0,0)] ## (LINE #, STRING, Match_Obj, Del_Obj)       
        segment = re.compile('[*****]')
        lookfor = re.compile('[COMMS_MESSAGE_SENT]')
        LineNum = 0
        
        for line in self.thefilesText:
            obj = lookfor.match(line)
            deli = segment.match(line)
            data.append((LineNum,line,obj,deli))
            LineNum += 1  
        
        return data

I could be going about this the wrong way. Should I be segmenting the log file fist (between delimiters), eg, 1 segment per list element
and then using RE to search the segments (then return the entire segment when found).

any input would be great!

  • 2 Contributors
  • 2 Replies
  • 180 Views
  • 9 Hours Discussion Span
  • Latest Post Latest Post by zyrus001

Dani AI

Generated

As pointed out, splitting the log into blocks and then checking each block for the pattern is the simplest, robust approach. Two practical variants follow: a streaming (memory‑friendly) generator for large files, and a quick split for smaller logs. Note that ’s original regex uses (square) character classes incorrectly and re.match() only checks the start of a line — re.search() or a simple substring test is usually what’s needed for “find anywhere in the block.”

A streaming generator (best for large logs; preserves low memory use):

import re

DELIM = re.compile(r'^\*+\s*$')     # line made only of asterisks
PAT   = re.compile(r'COMMS_MESSAGE_SENT')

def block_generator(f):
    buf = []
    for line in f:
        if DELIM.match(line):
            if buf:
                yield ''.join(buf)
            buf = []
        else:
            buf.append(line)
    if buf:
        yield ''.join(buf)

with open('log2.log', encoding='utf-8') as fh, open('matches.log', 'w', encoding='utf-8') as out:
    for block in block_generator(fh):
        if PAT.search(block):
            out.write(block)

A compact split-then-search (fine for small files):

import re
text = open('log2.log', encoding='utf-8').read()
blocks = re.split(r'(?m)^\*+\s*$', text)
for i, blk in enumerate(blocks):
    if 'COMMS_MESSAGE_SENT' in blk:
        print('segment', i)
        print(blk)

Troubleshooting notes: use re.search() (not re.match) to find substrings, escape * in regex or use \*/^\*+$, trim lines with strip() when comparing exact delimiters, and prefer substring checks ('PAT' in line) over regex when the match is literal — it’s faster and clearer. If line numbers are needed, enumerate the file and track the starting line of each block before yielding.

Recommended Answers

All 2 Replies

Member Avatar for jice

I'd store the blocks in a list, looking for the pattern in each line and, if i find the pattern in one line, write the whole block out.

isBlockToWrite=False
pattern="COMMS_MESSAGE_SENT"
block=[]
outfile=file("log3.log","w")
for line in file("log2.log"):
    block.append(line)
    if line[:10] == "*" * 10:
        if isBlockToWrite:
            outfile.writelines(block)
        block=[]
        isBlockToWrite=False
    if pattern in line:
        isBlockToWrite=True
Member Avatar for zyrus001

I was going about this the wrong way, thx so much!

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.