Help needed URL checking

[born2achieve]

Hi,

I am using windows7 64 bit. I have a text file "sample.txt" and text file has 200000 image url's like below


http://www.imagesup.net/img/icon_index1.png
http://www.imagesup.net/img/icon_index2.png

I wanted to check the image existson the directory, All my images are hosted remotely.Is it possible to achive using the batch script? if i executhe batch, it should output the url which doesn't have the image on output.txt file. Is it possible to achieve using DOS script. any sampel code please

Thanks

[ShadowThief]

It's possible, just not without third party tools, like cURL for Windows.

[foxidrive]

Using the spider option of wget seems doable: see http://stackoverflow.com/questions/6986 ... get-ing-it

[born2achieve]

Hey Guyz,
Thanks for your reply. I am not sure how to use the Wget. can you please help me how to use ?

[born2achieve]

To kick start, I tried like this

C:\> $ wget http://www.imagesup.net/img/icon_index6.png

Error:

'$' is not recognized as an inter operable program or batch file.

any help please

[born2achieve]

I downloaded the GnuWin32 and installed. after that i used below line. but still not hope to start. any help is appreciated

C:\>Program Files (x86)\GnuWin32\bin\wget.exe http://www.imagesup.net/img/icon_index8.png
'Program' is not recognized as an internal or external command,
operable program or batch file.

[ShadowThief]

To kick start, I tried like this

C:\> $ wget http://www.imagesup.net/img/icon_index6.png

Error:

'$' is not recognized as an inter operable program or batch file.

any help please

I'm guessing you saw the $ in a wget example. $ is the Linux prompt symbol, like how Windows has C:\>

I downloaded the GnuWin32 and installed. after that i used below line. but still not hope to start. any help is appreciated

C:\>Program Files (x86)\GnuWin32\bin\wget.exe http://www.imagesup.net/img/icon_index8.png
'Program' is not recognized as an internal or external command,
operable program or batch file.

If the path has spaces, you have to wrap the command in quotes. Use "Program Files (x86)\GnuWin32\bin\wget.exe"

[born2achieve]

Hi Shadow,

Thanks for your reply and am still having problem. then i tried the below


C:\>path = path + C:\Program Files (x86)\GnuWin32\bin\wget.exe
C:\Program Files (x86)\GnuWin32\bin>wget.exe http://www.imagesup.net/img/icon_index8.png

'wget.exe' is not recognized as an internal or external command,
operable program or batch file.

any help please

[born2achieve]

I downloaded the exe from the below path and installed

http://sourceforge.net/projects/gnuwin3 ... p_redirect

but i couldn't find the Wget.exe on the folder C:\Program Files (x86)\GnuWin32\bin

any clue? am doing something wrong?

[ShadowThief]

I downloaded the exe from the below path and installed

http://sourceforge.net/projects/gnuwin3 ... p_redirect

but i couldn't find the Wget.exe on the folder C:\Program Files (x86)\GnuWin32\bin

any clue? am doing something wrong?

The link you provided is only for downloading sed, which won't help you at all. Use http://sourceforge.net/projects/getgnuwin32/files/ to get the installation file.

[Squashman]

Hi Shadow,

Thanks for your reply and am still having problem. then i tried the below


C:\>path = path + C:\Program Files (x86)\GnuWin32\bin\wget.exe
C:\Program Files (x86)\GnuWin32\bin>wget.exe http://www.imagesup.net/img/icon_index8.png

'wget.exe' is not recognized as an internal or external command,
operable program or batch file.

any help please

That is not the proper syntax for adding to the path variable. If you echo the variable you should see that plain as day.

[foxidrive]

Thanks for your reply and am still having problem.

Please also tell us if you have solved the task.

[born2achieve]

Hey Guyz,

Finally i was able to install the GetGnuwin32 and i did all the installation specified in the document. Now i could see the Wget exe and i tried to

D:\GnuWin32\GetGnuWin32\bin> wget.exe http://pagead2.googlesyndication.com/si ... 4530698365

I could see the result on the command prompt and i can see the image downloaded on the root folder. Now Could you please help me on reading the URL.txt has 200000 urls and need to output which URL has doesn't have image on output.txt.

[Note: I don't want to download the image to my folder. I just need
get the URL which doesn't have image]

could you please help on making this process as batch script

Any help please.

[foxidrive]

Test this on a few URLs in url-list.txt in the current folder, and the file url.log will be written in the same folder.

Code: Select all

@echo off
(for /f "usebackq delims=" %%a in ("url-list.txt") do (
    "D:\GnuWin32\GetGnuWin32\bin\wget.exe" --spider "%%a" || echo missing %%a
))>url.log
pause

[born2achieve]

Hi Foxidrive,

Great and the code works fine, but am seeing below line on cmd and the i am seeing all my images are missing on output file though the image links are valid,

To connect to xxx.com insecurely, use `--no-check-certificate'.
(xxx - my website)

how to resolve this. any clue please