remove multiple instances from a textfile

[nouth]

this time I have only one textfile and I want to remove all duplicate lines so that each and every single line in it is unique unto itself.

with open("textfile") as w:
 for line in w:
  W = line
  with open("textfile") as d:
   c = 0
   for line in d:
    D = line
    if W.strip() == D.strip(): c = c + 1
    if c == >1:
     #how can I delete D?
     #and will after deleting it, does this mean that I can make sure that my W won't skip it?

[besticm]

where to use this coding..

Regards,

packers and movers in delhi

[tony75]

my oldtext.txt
"Which language do you like?
I like python programming.
Which language do you like?
I like python programming.
I like python programming."

oldFile = open("oldtext.txt")
newFile = open("newtext.txt", "w")
text = []
for line in oldFile:
    if line not in text:
        text.append(line)
        newFile.write(line)
newFile.close()

my newtext.txt
"Which language do you like?
I like python programming."

>>> ================================ RESTART ================================
>>> 
Which language do you like?

I like python programming.

[TrustyTony]

If the order of lines does not matter, you can use

  with open('filelein.txt') as infile,  open('fileout.txt', 'w') as outfile:
           outfile.write(''.join(set(infile))) 

[nouth]

I want to do it in such a way as to so that I can understand exactly what is happening.

with open(file) as f:
    for line in f:
        if i < 1:
            i = i + 1
        else:
            #I want to remove line from f

how can I delete line from f within for line in f?

okay, I got distracted with this

oldFile = open("oldtext.txt")
newFile = open("newtext.txt", "w")
text = []
for line in oldFile:
    if line not in text:
        text.append(line)
        newFile.write(line)
newFile.close()

I cannot understand what text = [] does and how it can work with if line not in text:
but can I change it to this?

if line not in text:
    text.append(line)
    oldFile.write(line)

or maybe I read something somewhere from about saving it to temporary files, so maybe newFile can be temporary and then it can be used with to reoverwrite the oldFile? no not like append

with open("textfile") as w:
    for line in w:
        W = line
        with open("textfile") as d:
            c = 0
            for line in d:
                D = line
                if W.strip() == D.strip(): c = c + 1
                if c == >1:
                    #I want to delete line in d
#but this is definitely the way that I want to do it, notice I am still opening the same "textfile" twice and then making a new variable for it's line

[snippsat]

so maybe newFile can be temporary and then it can be used with to reoverwrite the oldFile?

Yes this is normal way,you iterate over contend and do what you need,then save contend in a new file.
If you need newtext to be oldtext you rename the files.
os.rename(oldtext','newtext') or shutil.move('oldtext','newtext')

Python has fileinput that can do inplace edit of orginal file.
It's ok,but the method over with rename is just as good.
If order doesn't matter,soultion by pyTony is good.

'''lines.txt-->
line 1
line 2
line 2
line 3
'''

import fileinput
import sys

def remove_dup(fname):
    text = []
    for line in fileinput.input(fname, inplace=True):
        if line not in text:
            text.append(line)
            sys.stdout.write(line)

fname = 'lines.txt'
remove_dup(fname)

'''lines.txt-->
line 1
line 2
line 3
'''

[nouth]

I have learnt what text = [] does now
but I still have some trouble understanding this and how it works outfile.write(''.join(set(infile))) from pyTony

I haven't used import yet but can understand that for line in fileinput.input(fname, inplace=True): and sys.stdout.write(line) have something to do with import fileinput and import sys

but my next question I will ask will have to be about this

[TrustyTony]

"Write to outfile joined lines of set of items (lines) in file infile"
Set of values has only one instance of same item.