I need to download data automatically from a remote server. Its a CSV file, and i need to unzip it, and store the data in a mysql database that i created. There are multiple databases. The first section below is one of the databases that someone helped me with and it works wonderfully. I set up a cron job and it downloads it automatically as scheduled. However i have more databases and cant get the others to work.
THIS SECTION BELOW WORKS FINE!!!!
-----------------------------------------------------------------------------
#!/bin/bash
deleteparam='--delete-after';
#deleteparam=''
#directory='--directory-prefix=/$HOME/myfolder/myfolder/'
directory=''
#userpwd='--http-user=blah --http-password=blah'
toUpper() {
echo $1 | tr "[:lower:]" "[:upper:]"
}
#if [ -z "$1" ]; then
# echo usage: $0 table
#exit
#fi
ZIP=name of data file in remote server
ZIP=`toUpper $ZIP`
#the file within the zip has one less _
DATA=name of table in my mysql database
DATA=`toUpper $DATA`
#rm $TABLE.csv.zip
#rm $DATA.csv
cd /$HOME/myfolder/myfolder/
#######get the residential
wget 'http://3pv.mlslirealtor.com/Data3pv/DownloadBRSaction.asp?user_code=XXXXXX&password=XXXXXXX&data_type=datazip' -O $ZIP.csv.zip
sleep 10
#######unzip it
pwd
ls -la
unzip -o $ZIP.csv.zip
sleep 10
#######load it
wget --verbose $deleteparam $directory $userpwd http://mywebsite/myfolder/import.php?table=$DATA
sleep 300
######delete the files
rm $ZIP.csv.zip
rm $DATA.csvsleep 10
_---------------------------------------------------------------------------------
ALL OF THIS ABOVE WORKS WONDERFUL
THIS IS WHAT I NEED HELP WITH
----------------------------------------------------------------------------
1. I need a script to add to the above file that will download the photos. If you see below i need a script that automatically gets the CURRENT date and time that the script is ran. There server holds pic from the last 7 days and is constantly updated. Once this script is ran it will download the data into the table that i created.
2. Then, I need a script that queries the mysql database looking for entries that have photos and then retrieve the actual photos directly from their remote servers. This csv file DOES NOT download photos, just data that i can use to run a script to retrieve the photos at a given spot. See below.
Below are the instructions I received.
INSTRUCTIONS
Photo data is retrieved by HTTP. The photos data is updated once daily and is available for download as a CSV file. You can then write a script using the data from the CSV file to point back to the images on our image server. YOU MUST SUPPLY A VALUE FOR THE QUERY STRING last_updt > 'YYYYMMDD HH:MM:SS' for the URL to return data. The field last_updt is the date value for the last time that a photo was changed on the listing.
You will need to replace the user_code and password place holders (XXXX) with the login credentials provided to you.
Step 1: To retrieve the primary listing photo data CSV file go to the URL below.
http://remotewebserver/Data3pv/DownloadBRSaction.asp?user_code=XXX&password=XXXX&query_str=last_updt%20>%20'YYYYMMDD%20HH:MM:SS'&data_type=PHOTOS
Step 2: The photos table data should download in a CSV format.
Step 3: Using the Y flags and the ML Number in the data you can link back to our image locations.
Our primary image directory path is as follows:
http://remoteserver/folder/folder/folder/Last3DigistsofML#/listing#.jpg
For example- The primary photo for listing number 1899430 is located at
http://remoteserver/folder/folder/1/430/1899430.jpg
Our additional image directory path structure is as follows:
http://remoteserver/mlsphotos/full/PhotoPosition/Last3DigistsofML#/listing#_photoPosition.jpg
For example- The second photo for listing number 1899430 is located at