Member Avatar for sorin.oltean87

Hi,
We have a file (e.g. a .csv file, but could be any other format), with 2 columns: the old value and the new value. We need to modify all the files within the current directory (including subdirectories), so find and replace the contents found in the first column within the file, with the contents of the 2nd column

e.g. the file can contain:

COL1 -- COL2
old_one new_one

That 'new_one' string can contain '!' or single quotation marks, which i should escape them. Is it possible this? To get the value in shell script, from the first column (so, read the csv line by line), and the new value from the second column.
If so, how?

  • 2 Contributors
  • 4 Replies
  • 2K Views
  • 1 Day Discussion Span
  • Latest Post Latest Post by sorin.oltean87

Dani AI

Generated

Good question. The hardest part here is not sed itself, but quoting. In bash, the character ! is expanded by history when it appears inside double quotes, which is why your examples break. Two quick fixes if you stay with sed: either disable history expansion in the shell with set +o histexpand for the session, or put the sed program in single quotes and choose a delimiter that is not in your data (for example |). That way you are not fighting / or ! at all.

If you want something that handles quotes, commas, and embedded punctuation from a real CSV without hand-escaping, a tiny Python helper is more robust. It treats the CSV as data (column 1: old, column 2: new) and escapes patterns for you so replacements are literal, not regex:

#!/usr/bin/env python3
import csv, re, sys, pathlib

csv_path = sys.argv[1]
root = pathlib.Path(sys.argv[2] if len(sys.argv) > 2 else ".")

pairs = [(old, new) for old, new in csv.reader(open(csv_path, newline="")) if old]
keys = sorted({o for o, _ in pairs}, key=len, reverse=True)
lookup = dict(pairs)
rx = re.compile("|".join(re.escape(k) for k in keys))

for p in root.rglob("*"):
    if not p.is_file():
        continue
    data = p.read_bytes()
    if b"\0" in data:
        continue  # skip binaries
    text = data.decode("latin-1")  # preserves bytes 1:1; use utf-8 if you know files are utf-8
    new = rx.sub(lambda m: lookup[m.group(0)], text)
    if new != text:
        p.write_bytes(new.encode("latin-1"))

Usage:

  • python3 replace_from_csv.py mapping.csv ./server

Notes:

  • Sorting by longest key first avoids partial overlaps.
  • re.escape means characters like ', !, *, . are matched literally.
  • If you continue with ’s sed-script idea, remember to escape regex metacharacters on the left side and & and \ on the right side, and keep history expansion off.

Recommended Answers

All 4 Replies

Member Avatar for nezachem

Step 1: form a sed script based on the csv file
Step 2: apply sed to all the files you need to modify

Assuming csv (that is, columns are separated with comma, and comma doesn't appear anywhere neither in old nor in new values), the sed script can be trivially formed with a following sed command:

sed -e 's/\(.*\)/s,\1,g/' file.csv script.sed

Now you may apply it as

for file in list_of_files; do sed -i -f script.sed $file; done

I didn't get the escaping part (what and how should be escaped), but it is surely possible.

Member Avatar for sorin.oltean87

I didn't et the escaping part (what and how should be escaped), but it is surely possible.

OK, let's say in the first column i can have "It's too late for this action!", and in the second (replacing string), "Some text here".

Now, when replacing, how can i automatically add the escape "\" character, in front of special characters, when parsing "It's too late for this action!", to find this string in files, and then replace it?

What does exactly this?

sed -e 's/(.*)/s,\1,g/' file.csv script.sed

Thanks

Member Avatar for nezachem

Literally it is an instruction to sed to prepend a line with s, and append ,g to it.
Say, some line in the replacement file reads
foo,bar
Then, after the sed application it'll become
s,foo,bar,g
which in turn is a sed instruction to replace (s) all (globally) appearances of foo to bar.

To escape bangs you may also use sed, this time with s/!/\\!/g instruction.

Member Avatar for sorin.oltean87

Thanks for reply.
I decided to write sepparate commands for each file that will be corrected.
For example i have this command:

find ./server -type f -name 'CERINTE_BU.sql' -exec sed -i "s/raise_application_error(-20998, 'Error text/please recheck!');/os_err.raise_err('ERR_1472');--'Error text/please recheck!'/g" {} \;

Well, so raise_application_error(-20998, 'Error text/please recheck!'); should be replaced with os_err.raise_err('ERR_1472');--'Error text/please recheck!'

In this command, what option can i specify and where, to automatically add "\" in front of specal characters?

I tried in this way, but it's not working:

find ./server -type f -name 'CERINTE_BU.sql' -exec sed -i "s/!//\\!/graise_application_error(-20998, 'Error text/please recheck!!');/os_err.raise_err('ERR_1472');--'Error text/please recheck!'/g" {} \;

So in this case, for example, escape characters should be added in front of the every "!", and other special characters

Thanks

Edited by sorin.oltean87
Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.