How to run multiple python file in a folder one after another

[Karthik_4]

Hi

I have around 20 python files. Each time I run these files in the terminal this form one after another :
python a.py
python b.py
python c.py
python d.py
python e.py
python f.py
python g.py
.
.
.
(I have provided general file names here)

This process takes lot of time.
Is it possible to run these file together one after another through any script..?
If possible, then how..?
Please provide the code if possible...
I came to know through few sites that, using bash script we can do that..
I don't know how to implement it.
I am working on ubuntu14.04

[AndrisP]

You can put all in a shell script file e.g. "run_py_files.sh"

/full/path/to/python/a.py
/full/path/to/python/b.py
/full/path/to/python/c.py
/full/path/to/python/d.py
/full/path/to/python/e.py
/full/path/to/python/f.py
/full/path/to/python/g.py

make it executable and call only one shell script

[Karthik_4]

yeah.. but where is the python command, to execute ??

We need python /full/path/to/python/a.py to execute right ?! inside shell script.

[AndrisP]

You dont need python command. Make shell script executable and python scripts executable.
e.g. if you put in terminal cd /full/path/to/python "Enter" and then put a.py "Enter" then do not work but if shell record starts by slash like /full/path/to/python/a.py then it call as executable file

[Gribouillis]

You can get the path to the python command by pasting this in a terminal:

python -c "import sys; print(sys.executable)"

edit: @AndrisP what you say will only work if the python scripts start with the correct shebang line containing the call to the python executable, unless the system has been configured to recognize .py files as python scripts (which brings new issues, such as what to do if one has multiple versions of python).

[AndrisP]

@Gribouillis Actually no its easy way to call python scripts from shell script file. Versions of Python declare in a .py file starts like #!/usr/bin/python or #!/usr/bin/python3 or other

[Karthik_4]

what if I need to take seperate nosetests html report for each file.. How can I do that ?
Because, nosetests a.py --with-html-out generates results.html.

Running the nosetests command in shell script for all python file will create one 'results.html'.
I get the report of the last python file in shell script.. Because all other reports were overriten. (the name was same. Thats why!!!)

[Gribouillis]

@AndrisP yes I know. What you describe is called the shebang. You can also use

#!/bin/env python

which uses the environment to find the python path.

I was thinking about another mechanism called binfmt_misc in the linux kernel: if you type

sudo update-binfmts --install Python /usr/bin/python --extension py

Then files with extension .py will automatically be executed with /usr/bin/python (without the need of a shebang line). The problem is that nowadays, both python and python3 source files end with .py, so that binfmts_misc is more annoying than useful in this case.

[Gribouillis]

@OP don't you have an option --html-file ?

[Karthik_4]

I never tried that.. Can I get more info on --html-file ??
How to use it ??

[Gribouillis]

No, I don't know, I thought that you were using a plugin such as Click Here or Click Here which have this option. You could perhaps try them.

[JasonHippy]

I'm not at all familiar with the nosetests package, I've never used it.
And from taking a look at its man-page online, I can't see anything that mentions a --html-file option.

However, I do know shell scripting. So here's a quick, simple shell-script to execute all .py files in the current directory, which renames the results.html file to filename.py.html - where filename.py is the name of the python file being tested.
runtests.sh

#!/bin/bash

#default html file output by nosetests
OUTFILE=results.html 

for FILE in ./*.py;
do
    nosetests $FILE --with-html-out # run test
    if [ -f $OUTFILE ]; then     # if results.txt exists
        mv $OUTFILE $FILE.html   # rename it to testfile.py.html
    fi
done

The above script would have to be in the same directory as the python files you are testing. Make sure you set the execute permissions-bit on the script (chmod +x ./runtests.sh), then you can use ./runtests.sh to run the tests.

After running the script, you should see a bunch of html files alongside your python files, containing the results of the tests.

That is just a simple script that I knocked together quickly. If you want to make it more useful - it would be trivial to modify the script to take a path to a directory as a parameter. That way you could move the script into /home/yourusername/bin/ and you could use it as a general tool to run tests on ANY directory containing python files.

You could modify it further to use something like find to recursively find and run tests on python files in all of the sub-directories in the given directory.

Finally, from a quick look at nosetests documentation, it looks like you could also write python scripts that hook into the nosetests framework to run your tests too. But I don't have time to try to work out how to use nosetests ATM.... Sorry! :)

[Karthik_4]

Exactly.. Even I tried the same thing now..

#!/bin/bash

for f in *.py
do 
    echo "$f"
    nosetests "$f" --with-html --html-file="$f".html  
done

This works fine..

Thanks a lot for the help people..

[snippsat]

what if I need to take seperate nosetests html report for each file.. How can I do that ?

Yes as bash script looping over files will work,or a Python script with subprocess.
There are build into nose to test all files that start test.
So naming your files test_a.py test_b.py... will work.

But this give you on report at end,
there are soultion like tappy

I use pytest,not tested but this should work,
py.test --files --html=report.html test all file in a folder,
and each files get own report.

[Karthik_4]

#!/bin/bash
for f in *.py
do 
    echo "$f"
    nosetests "$f" --with-html --html-file="$f".html  
done

What happens here is, it runs all the .py files and creates a html report with respect to those files in the same directory.

I need to create a seperate folder name with the name date and time, to store the report. How can I store the html files in seperate folder once it is created ?

[Gribouillis]

Did you try --html-file="myfolder/${f}.html" ?

[Karthik_4]

No..
I found a work around for that..
First I found the files with extension .html and then created a folder with timestamp. Then moved the files inside the folder.. :)