AutoCheck webpage for new content

[julesverne]

Hi all,

The next solution I'm hoping to find is to automagically check for new content on a website. I don't mind including a 3rd party command line executable in the bat file, I'm just interested in doing this in bat form so that i can have it check periodically every hour, without me having to remind myself.

This site does not have an rss feed so that won't work. It's a job posting site.

Thanks for any help!

Jules

[npocmaka_]

Is using .NET/Vbscript/Jscript/powershell considered as a cheating?
Without another language your only options is BITSADMIN , but the command is not avaialable on all windows systems.

[julesverne]

@npocmaka_ no, those aren't cheating for me if it accomplishes my goal. I'm unfamiliar with scripting in those options so any help you'd offer would probably be 100% from your end as I wouldn't be able to offer anything. If you do have a solution that lies outside of dostips forum standards, please PM me with what you have. If we come with up a solution that works I'll post "solved by alternate code solution here" so people know the problem is solved and we're not breaking any forum rules, that I'm aware of. That being said, if an admin believes otherwise, please let me know. I like this board and don't want to be booted from it.

[foxidrive]

What is 'new content' ?

The most simplistic comparison would be to get the web page, and compare it using FC on the next download in say 30 minutes time.

If that works for you then a VBS script can download the web page, or you can use WGET.

[npocmaka_]

What is 'new content' ?

The most simplistic comparison would be to get the web page, and compare it using FC on the next download in say 30 minutes time.

If that works for you then a VBS script can download the web page, or you can use WGET.

Most of the examples with vbscript use a hidden internet explorer and I don't like this approach as it is too heavy.
You can also check this implementation of cURL with jscript ->https://code.google.com/p/curlie/


and here's one way with jscript.net :

Code: Select all

@if (@X)==(@Y) @end /************

@echo off

set "page_to_check=http://www.dostips.com/"


::::::::::::::::::::::::::::::::::::
:::       compile the script    ::::
::::::::::::::::::::::::::::::::::::
setlocal
if exist simpledownloader.exe goto :skip_compilation

set "frm=%SystemRoot%\Microsoft.NET\Framework\"
:: searching the latest installed .net framework
for /f "tokens=* delims=" %%v in ('dir /b /s /a:d /o:-n "%SystemRoot%\Microsoft.NET\Framework\v*"') do (
   if exist "%%v\jsc.exe" (
      rem :: the javascript.net compiler
      set "jsc=%%~dpsnfxv\jsc.exe"
      goto :break_loop
   )
)
echo jsc.exe not found && exit /b 0
:break_loop


call %jsc% /nologo /out:"simpledownloader.exe" "%~dpsfnx0"
::::::::::::::::::::::::::::::::::::
:::       end of compilation    ::::
::::::::::::::::::::::::::::::::::::
:skip_compilation


simpledownloader.exe "%page_to_check%" test_file1.txt


:wait_label
rem :: this will sleep for 10 seconds
w32tm /stripchart /computer:localhost /period:1 /dataonly /samples:10  >nul 2>&1


simpledownloader.exe "%page_to_check%" test_file2.txt

FC test_file1.txt test_file2.txt | FIND "FC: no dif" > nul 
IF ERRORLEVEL 1 (
   echo page has been updated
   copy /Y test_file2.txt test_file1.txt
   del /Q /F test_file2.txt
)
rem set errorlevel to 0
color

goto :wait_label



****** end of jscript comment ******/

import System;
var arguments:String[] = Environment.GetCommandLineArgs();
var webClient:System.Net.WebClient = new System.Net.WebClient();
print("Downloading " + arguments[1] + " to " + arguments[2]);
try {
   webClient.DownloadFile(arguments[1], arguments[2]);
} catch (e) {

      Console.BackgroundColor = ConsoleColor.Green;
      Console.ForegroundColor = ConsoleColor.Red;
      Console.WriteLine("\n\nProblem with downloading " + arguments[1] + " to " + arguments[2] + "Check if the internet address is valid");
      Console.ResetColor();
      Environment.Exit(5);
}

This will download the index page of a internet adress to a text file and will compare it with its previous version.

[Samir]

What is 'new content' ?

The most simplistic comparison would be to get the web page, and compare it using FC on the next download in say 30 minutes time.

If that works for you then a VBS script can download the web page, or you can use WGET.

This was my thinking since wget is good for downloading sites. Plus, I believe it has an 'update' feature which will only download the newest files. You can probably combine this with some errorlevel checking to alert you.