multiple file argument

[weblover]

hello all,

i'm new to python programming and i'm facing a problem and need ur help

i finished my script and i'm trying to execute it on windows cmd

the script takes a file as an argument , when i run the script with the argument it works very well , but what i want to do is : i want the script to take the whole folder containing these files as an argument and calculate the result for them all.

i tried the following :

C:\Python27\mtdb\src\refseq>python ../skew.py -e"A" *.gb

but it didn't work

thanks in advance.

[Gribouillis]

Command line arguments must be parsed and handled in your script. A common way to do this is to use the optparse module. This blog entry will teach you enough of optparse for your needs http://www.alexonlinux.com/pythons-optparse-for-human-beings .

[weblover]

hello ,

thanks for your reply , but i'm new to python and i already used this module but i did not know how to integrate the multiple argument in my case .

and also when i try to delete multiple files on windows :

Del *.gb

it's not working , maybe the error is from the windows ,i don't know.

if you want i'll show my code and you can help me.

thanks in advance

[Gribouillis]

hello ,

thanks for your reply , but i'm new to python and i already used this module but i did not know how to integrate the multiple argument in my case .

and also when i try to delete multiple files on windows :

Del *.gb

it's not working , maybe the error is from the windows ,i don't know.

if you want i'll show my code and you can help me.

thanks in advance

Yes, it's a very good idea to post your code.

[weblover]

ok i will post my code, but now i tried the

DEL *.gb

and it worked on cmd

so the problem is from the script

from optparse import OptionParser, OptionGroup
import os.path
import random
import sys

from gb import gbfromfile
from sequence import sequence, randsequence, sequences_fromfile

usage = "usage: %prog [options] file"
parser = OptionParser( usage )
parser.add_option( "-e", "--expr", action = "append", type = "string", metavar = "EXPR", help = "add expression to evaluate and print" )
parser.add_option( "--third", action = "store_true", default = False, help = "only consider third codon position" )

parser.add_option( "-f", "--format", action = "store", type = "string", metavar = "FMT", help = "output format FMT: svm" )
parser.add_option( "-o", "--outfile", action = "store", type = "string", metavar = "FILE", help = "write values to FILE (default: stdout)" )
parser.add_option( "-r", "--random", action = "store", type = "string", metavar = "MODE", help = "generate random subsequences of the same length, valud MODEs: shuffle & random" )
parser.add_option( "-c", "--circular", action = "store_true", default = False, help = "treat sequence(s) as circular (doeas not apply to gb files)" )

group = OptionGroup( parser, "select a subset of the features in the genbank file for calculations. "
                "(type can be tRNA, rRNA, and gene) "
                "Note: -y,-Y,-n,-N can be specified more than once, combinations are possible." )
group.add_option( "-y", "--atype", action = "append", type = "string", metavar = "TYPE", help = "get all features of type TYPE" )
group.add_option( "-Y", "--ftype", action = "append", type = "string", metavar = "TYPE", help = "get all features except features of type TYPE" )

group.add_option( "-n", "--aname", action = "append", type = "string", metavar = "NAME", help = "get all features with name NAME" )
group.add_option( "-N", "--fname", action = "append", type = "string", metavar = "NAME", help = "get all features except features with name NAME" )

group.add_option( "-s", "--strand", action = "store", type = "int", metavar = "STRAND", help = "get values for +/- strand" )
parser.add_option_group( group )

parser.add_option( "-t", dest = "atax", action = "append", type = "string", metavar = "TAX", help = "allow only entries with TAX in the taxonomy" )
parser.add_option( "-T", dest = "ftax", action = "append", type = "string", metavar = "TAX", help = "forbid all entries with TAX in the taxonomy" )
parser.add_option( "-w", "--window", action = "store", type = "int", metavar = "SIZE", help = "use windows of size SIZE" )
parser.add_option( "--offset", action = "store", type = "int", default = 1, metavar = "OFF", help = "set offset to OFF" )

( options, args ) = parser.parse_args()

that's not the full code because the full code is very large , so if you can help me with this part and if no i will post the rest in another comment

thnx in advance

[Gribouillis]

ok i will post my code, but now i tried the

DEL *.gb

and it worked on cmd

so the problem is from the script

from optparse import OptionParser, OptionGroup
import os.path
import random
import sys

from gb import gbfromfile
from sequence import sequence, randsequence, sequences_fromfile

usage = "usage: %prog [options] file"
parser = OptionParser( usage )
parser.add_option( "-e", "--expr", action = "append", type = "string", metavar = "EXPR", help = "add expression to evaluate and print" )
parser.add_option( "--third", action = "store_true", default = False, help = "only consider third codon position" )

parser.add_option( "-f", "--format", action = "store", type = "string", metavar = "FMT", help = "output format FMT: svm" )
parser.add_option( "-o", "--outfile", action = "store", type = "string", metavar = "FILE", help = "write values to FILE (default: stdout)" )
parser.add_option( "-r", "--random", action = "store", type = "string", metavar = "MODE", help = "generate random subsequences of the same length, valud MODEs: shuffle & random" )
parser.add_option( "-c", "--circular", action = "store_true", default = False, help = "treat sequence(s) as circular (doeas not apply to gb files)" )

group = OptionGroup( parser, "select a subset of the features in the genbank file for calculations. "
                "(type can be tRNA, rRNA, and gene) "
                "Note: -y,-Y,-n,-N can be specified more than once, combinations are possible." )
group.add_option( "-y", "--atype", action = "append", type = "string", metavar = "TYPE", help = "get all features of type TYPE" )
group.add_option( "-Y", "--ftype", action = "append", type = "string", metavar = "TYPE", help = "get all features except features of type TYPE" )

group.add_option( "-n", "--aname", action = "append", type = "string", metavar = "NAME", help = "get all features with name NAME" )
group.add_option( "-N", "--fname", action = "append", type = "string", metavar = "NAME", help = "get all features except features with name NAME" )

group.add_option( "-s", "--strand", action = "store", type = "int", metavar = "STRAND", help = "get values for +/- strand" )
parser.add_option_group( group )

parser.add_option( "-t", dest = "atax", action = "append", type = "string", metavar = "TAX", help = "allow only entries with TAX in the taxonomy" )
parser.add_option( "-T", dest = "ftax", action = "append", type = "string", metavar = "TAX", help = "forbid all entries with TAX in the taxonomy" )
parser.add_option( "-w", "--window", action = "store", type = "int", metavar = "SIZE", help = "use windows of size SIZE" )
parser.add_option( "--offset", action = "store", type = "int", default = 1, metavar = "OFF", help = "set offset to OFF" )

( options, args ) = parser.parse_args()

that's not the full code because the full code is very large , so if you can help me with this part and if no i will post the rest in another comment

thnx in advance

Ok, I added a a few lines to your code to see what it does:

print("parsed options are\n")
print(vars(options))
print("remaining arguments are\n")
print(args)


""" example session in a (linux) terminal. It should look the same in cmd -->

$ skew.py -n foo -N bar qsdf iouyzer lkjze zerr
parsed options are

{'ftax': None, 'third': False, 'format': None, 'expr': None, 'random': None, 'atype': None, 'atax': None, 'outfile': None, 'window': None, 'ftype': None, 'fname': ['bar'], 'offset': 1, 'aname': ['foo'], 'strand': None, 'circular': False}
remaining arguments are

['qsdf', 'iouyzer', 'lkjze', 'zerr']

"""

Only the code in your previous post does nothing but parse the arguments. Can you give an example of a real command line and describe the expected effect ? You can also post the rest of the module (note that I don't have the gb and sequence modules).

[weblover]

hello,

i typed this command line :

python ../skew.py -e"A" NC_000834.gb

and got this output :

c:\python27\lib\site-packages\Bio\utils.py:17: BiopythonDeprecationWarning: Bio.
utils has been deprecated, and we intend to remove it in the next release of Bio
python.
  "the next release of Biopython.", Bio.BiopythonDeprecationWarning)
parsed options are

{'ftax': None, 'third': False, 'format': None, 'expr': ['A'], 'random': None, 'a
type': None, 'atax': None, 'outfile': None, 'window': None, 'ftype': None, 'fnam
e': None, 'offset': 1, 'aname': None, 'strand': None, 'circular': False}
remaining arguments are

['NC_000834.gb']
c:\python27\lib\site-packages\Bio\Seq.py:134: BiopythonDeprecationWarning: Acces
sing the .data attribute is deprecated. Please use str(my_seq) or my_seq.tostrin
g() instead of my_seq.data.
  "my_seq.data.", Bio.BiopythonDeprecationWarning)
0.268912

and the red number is the result that should be shown

but when i tried to calculate this number to all the gb files directly like this :

python ../skew.py -e"A" *.gb

i got this output :

c:\python27\lib\site-packages\Bio\utils.py:17: BiopythonDeprecationWarning: Bio.
utils has been deprecated, and we intend to remove it in the next release of Bio
python.
  "the next release of Biopython.", Bio.BiopythonDeprecationWarning)
parsed options are

{'ftax': None, 'third': False, 'format': None, 'expr': ['A'], 'random': None, 'a
type': None, 'atax': None, 'outfile': None, 'window': None, 'ftype': None, 'fnam
e': None, 'offset': 1, 'aname': None, 'strand': None, 'circular': False}
remaining arguments are

['*.gb']
Traceback (most recent call last):
  File "../skew.py", line 95, in <module>
    gb = gbfromfile( arg )
  File "C:\Python27\mtdb\src\gb\__init__.py", line 136, in __init__
    gbhandle = open( gbfile, "r" )
IOError: [Errno 22] invalid mode ('r') or filename: '*.gb'

what to do ?

[Gribouillis]

hello,

i typed this command line :

python ../skew.py -e"A" NC_000834.gb

and got this output :

c:\python27\lib\site-packages\Bio\utils.py:17: BiopythonDeprecationWarning: Bio.
utils has been deprecated, and we intend to remove it in the next release of Bio
python.
  "the next release of Biopython.", Bio.BiopythonDeprecationWarning)
parsed options are

{'ftax': None, 'third': False, 'format': None, 'expr': ['A'], 'random': None, 'a
type': None, 'atax': None, 'outfile': None, 'window': None, 'ftype': None, 'fnam
e': None, 'offset': 1, 'aname': None, 'strand': None, 'circular': False}
remaining arguments are

['NC_000834.gb']
c:\python27\lib\site-packages\Bio\Seq.py:134: BiopythonDeprecationWarning: Acces
sing the .data attribute is deprecated. Please use str(my_seq) or my_seq.tostrin
g() instead of my_seq.data.
  "my_seq.data.", Bio.BiopythonDeprecationWarning)
0.268912

and the red number is the result that should be shown

but when i tried to calculate this number to all the gb files directly like this :

python ../skew.py -e"A" *.gb

i got this output :

c:\python27\lib\site-packages\Bio\utils.py:17: BiopythonDeprecationWarning: Bio.
utils has been deprecated, and we intend to remove it in the next release of Bio
python.
  "the next release of Biopython.", Bio.BiopythonDeprecationWarning)
parsed options are

{'ftax': None, 'third': False, 'format': None, 'expr': ['A'], 'random': None, 'a
type': None, 'atax': None, 'outfile': None, 'window': None, 'ftype': None, 'fnam
e': None, 'offset': 1, 'aname': None, 'strand': None, 'circular': False}
remaining arguments are

['*.gb']
Traceback (most recent call last):
  File "../skew.py", line 95, in <module>
    gb = gbfromfile( arg )
  File "C:\Python27\mtdb\src\gb\__init__.py", line 136, in __init__
    gbhandle = open( gbfile, "r" )
IOError: [Errno 22] invalid mode ('r') or filename: '*.gb'

what to do ?

You must use module glob to expand the regular expression '*.gb" to a list of files with extension.gb

import glob
list_of_filenames = glob.glob("*.gb")

Then it's up to you to handle the fact that you don't have a single file name but a whole list of them.

[weblover]

where should i type this ??

[Gribouillis]

where should i type this ??

Well, in your program, after the parse_args, if the remaining args are supposed to be
file expressions like *.gb, you could add

import glob
for arg in args:
    filenames = glob.glob(arg)
    # here do something with the list of filenames

Alternately you could write

filenames = list(f for arg in args for f in glob.glob(arg))

Edit: changed the previous line of code.

[weblover]

:S:S it did not worked ,

that's the code where i should post the code , but it's not working

for arg in args:
    gb = None
    seq = None
    base, ext = os.path.splitext( arg )
    if ext == ".gb":
        gb = gbfromfile( arg )
        if not gb.is_allowed( options.atax, options.ftax ):
            continue
        seq = gb.sequence
        circ = gb.circular

    elif ext == ".fas" or ext == ".fa":
        circ = options.circular
        seq = sequences_fromfile( arg, circular = circ )[0]

thanks

[Gribouillis]

:S:S it did not worked ,

that's the code where i should post the code , but it's not working

for arg in args:
    gb = None
    seq = None
    base, ext = os.path.splitext( arg )
    if ext == ".gb":
        gb = gbfromfile( arg )
        if not gb.is_allowed( options.atax, options.ftax ):
            continue
        seq = gb.sequence
        circ = gb.circular

    elif ext == ".fas" or ext == ".fa":
        circ = options.circular
        seq = sequences_fromfile( arg, circular = circ )[0]

thanks

Try this

filenames = list(f for arg in args for f in glob.glob(arg))
print(filenames)

Does it print the list of files that your are targeting with your "*.gb" ?

(byt the way "it doesn't work" is not a very useful description of your issue. Why doesn't it work ?)

[weblover]

sorry ,

when i added these 2 lines inside the for loop ,
i get this error :

C:\Python27\mtdb\src\refseq>python ../skew.py -e"A" *.gb
  File "../skew.py", line 87
    filenames = list(f for arg in args for f in glob.glob(arg))
            ^
IndentationError: expected an indented block

[Gribouillis]

sorry ,

when i added these 2 lines inside the for loop ,
i get this error :

C:\Python27\mtdb\src\refseq>python ../skew.py -e"A" *.gb
  File "../skew.py", line 87
    filenames = list(f for arg in args for f in glob.glob(arg))
            ^
IndentationError: expected an indented block

Can't you post the whole code, starting from the line (options, args) = parser.parse_args() ? If it is too long, you can simply zip the python file and attach the zip to a post.

[weblover]

ok ,

( options, args ) = parser.parse_args()
if len( args ) == 0:
    sys.stderr.write( "no input file(s) given\n" )
    print usage
    sys.exit()

# nothing to evaluate
if options.expr == None:
    sys.stderr.write( "no expression given\n" )
    print usage
    sys.exit()

if options.random != None and options.random != "random" and options.random != "shuffle":
    sys.stderr.write( "invalid mode for random: %s\n" % options.random )
    sys.exit()

# outfile and outdir given ?
if options.outfile == None:
    ohandle = sys.stdout
else:
    ohandle = open( options.outfile, "w" )

if random != None:
    random.seed()



for arg in args:
      
    gb = None
    seq = None
    base, ext = os.path.splitext( arg )
	
    if ext == ".gb":
        gb = gbfromfile( arg )
        if not gb.is_allowed( options.atax, options.ftax ):
            continue
        seq = gb.sequence
        circ = gb.circular
	   
      
    elif ext == ".fas" or ext == ".fa":
        circ = options.circular
        seq = sequences_fromfile( arg, circular = circ )[0]

    subs = []
    if options.window == None and gb == None:
        subs.append( [ [seq.subseq( 0, len( seq ), options.strand ), options.strand] ] )

thanks in advance

[Gribouillis]

Ok, then replace the first line of your post with

( options, args ) = parser.parse_args()
import glob
args = list(f for arg in args for f in glob.glob(arg)) # expand regex in args

You can add a print(args) afterwards to see what it did to args.

[weblover]

thank you alot ,

it worked :D

how can i give you points ?

you helped me alot :D

[Gribouillis]

thank you alot ,

it worked :D

how can i give you points ?

you helped me alot :D

Thanks. If you want to give me points, you click an up arrow on the post which helps you the most, and then the "add reputation" button :)

[weblover]

just 1 more question ,

how can i output the result to a text file or excel sheet because it's very huge to be seen in the CMD

[Gribouillis]

just 1 more question ,

how can i output the result to a text file or excel sheet because it's very huge to be seen in the CMD

From what you posted above, it looks that the best way is to use your program's -o option.

[weblover]

oh sorry :| , i forgot that i added these feature :)

btw thanks alot for your help ,