First use a hidden frame:
<FRAMESET ROWS="100%,*"> <FRAME NAME="upper" SRC="http://somewhere.com/apage.html"> <FRAME NAME="lower" SRC="http://somewhere.com/"> </FRAMESET>
Then in the document in the upper frame, you read all the links in the bottom frame using the links array.
Once you have all the links, you then proceed to load each link in the lower frame. Once loaded you check the lastModified date, and hold that against the name of the file. You then move onto the next document...
Several hours and much bandwidth later you should have a complete list of all the files in the top level directory and their lastModified dates. You can then output this list ino the upper frame.
You could maitain a list of the documents and then hardcode the date that you last amended them.
You can then combine this with cookies and the current date to highlight which documents are new.