Question: grep looping over two file types

[llocar]

Hi.

I'm wanting to append some information to a file, which is found in a different file. I have simplified the problem using animals.

Here is the basic code to add the line to the file:

grep "some important info about rabbits" rabbits.data >> rabbits.txt

Very simple, but this must be repeated over lots of pre-existing files, e.g. for birds (birds.data, birds.txt), bees (bees.data, bees.txt), beetles (beetles.data, beetles.txt) etc, etc.

The files always have the same prefix (e.g. rabbits), but the file suffixes are different (.data, .txt).

The files must match - only info from rabbits.data should go into rabbits.txt etc

I'm sure this must be wrapped in a "for" loop, but I've had no luck so far, even after reading several tutorials and searching the forum. Any pointers in the right direction are greatly appreciated.

Thanks.

[griswolf]

Something resembling this:

for srcfile in $(echo *.data) ; do
  bn=$(basename $srcfile .data)
  sinkfile={$bn}.txt
  grep "something about $bn" srcfile >> sinkfile
done

Assuming bash, but ksh should work also. The $(...) becomes `...` in sh (Bourne shell)
line 1: echo *.data works only if you are in the correct directory. You can also use find . *.data (but beware using basename in that case)
line 1: the trailing ; do can be removed and a solo do placed on the next line if you prefer that style
line 2: You will want to use bn=${srcfile%.data} so as to keep the path if you need to work multiple directories, use find or whatever

I believe the rest is 'obvious'?

Addendum: If the initial data is valuable or hard to recover, it is well to make a copy of the sinkfile then redirect the output onto the original, leaving a pristine copy available in case of trouble. Beware that if there is trouble you must first rename the copy to the original name before you try again. Yes, I still have scars...

[llocar]

Thanks so much for the prompt help.

I ran the following, but ended up with a error: "grep: srcfile: No such file or directory".

for srcfile in `echo *.data` ; do
bn=`basename $srcfile .data`
sinkfile={$bn}.txt
grep "some text $bn" srcfile >> sinkfile
done

Changing srcfile in the grep line to $srcfile resulted in no error, but just an empty sinkfile.

Hmmm.

[griswolf]

Woops. I see that I've been doing so much in Python that my bash is starting to erode a bit.

Line 4 needs $ signs on $srcfile and $sinkfile

If that doesn't work, just pull it apart and work on each piece until it works, then stitch it back together.

[llocar]

Great. Thanks ever so much. I finally got it working with your help.

It seemed the $bn inside the grep "..." wasn't necessary.

I used:

for srcfile in `echo *.data`
do
bn=`basename $srcfile .data`
sinkfile=$bn.txt
grep "some text" $srcfile >> $sinkfile
done

As you say, writing straight into my original files is a bad idea, so I will be sure to make backup copies before "going global".

Cheers.

[griswolf]

When a thread is solved, the OP (Original Poster) should go find the "thread is solved" link at the bottom of the page to mark it solved. Helps keep Daniweb functioning as well as possible, and gives a small amount of credit to the folks who helped you. Nobody else has that link, so it's up to you.

PS: I find it useful to use indentation to help my eye track loop blocks, if/fi blocks and similar. I suggest you consider writing your scripts that way too.

[llocar]

Great. Will do.

Thanks for the advice on the indents.