webStore

webStore is an app for windows phones which lets you automatically download and store sets of web pages on your phone so that you can browse them later without any internet connection.

webStore can be downloaded here: http://www.windowsphone.com/en-gb/store/app/webstore/99277ab3-92b3-43ac-a5b6-6400d53f154b

webStore should be useful for two kinds of sites. For news sites, one can download a set of pages every day. It can also be used for storing pages with useful information which one may need to refer to from time to time, perhaps consisting of an entire website.

General usage

webStore stores a list of sets of pages to be downloaded and read. Each set has a starting URL and from there the app will pursue links down to a certain depth. A depth of 1 means the app just downloads the specified page. A depth of 2 means it also downloads every page the first one links to, and so on. It is only intended to follow simple links and anything complicated will break it.

You will usually want to make sure that the app does not end up following some links, for example those that go off the starting site. To achieve this you can use an include list and an exclude list. The include list is a set of strings of characters, one string per line, and the URL of a link must include one of these strings if it is to be followed. For example, to keep the app downloading only links from the news part of the BBC website you might specify uk/news as an include string, with http://m.bbc.co.uk/news as the starting URL. Likewise, if the link contains any strings from the exclude list then it will not be followed, so for example you could specify .mpg to prevent videos being downloaded. The app shows you which files it is downloading so this can give you an idea of which strings to specify in order to follow only the links you want to. Another important way to get a better idea of which criteria to use is to view the site in a desktop browser and view the HTML source of the page (generally right-click, view source). For example, to get the periodic table example to work I had to exclude plugin and also html/feed because I found that there were links called things like .../hydrogen.html/feed which would stop .../hydrogen.html from being saved properly. The lists are case sensitive.

To get the URL for a page, you can navigate to a site using a browser and then copy the URL and paste it into webStore. Then you can tweak the download parameters. Or you can set up the required URLs on your desktop and then email them to your phone and copy and paste them from there.

You can give the saved set a convenient name or else it will just use the starting URL.

You can specify a number of days after which the set will expire. Then you can download all expired sets at once from the main menu. If you specify 0 days then the set will always be downloaded. For rolling news sites, one might want to set this value to zero. For newspapers, one might set it to 1. For a journal or blog one might set it to 7 and for reference sites one might leave it blank so that they were never downloaded automatically but only when specifically requested.

Maintenance operations

Before a set is downloaded, all the pages previously saved for that set are deleted so the amount of space required should not grow continuously. If the specifications for a set are deleted then all associated saved pages are also deleted. If there are problems and the space used by the app's files seems to be growing or if the sets become corrupted there is an option to delete all saved content.

You can backup your set definitions by copying them to the clipboard and then, for example, pasting them into a page of OneNote of into an email which you send to yourself. Then, if you ever need to restore the definitions in the future you can first delete all saved content and then copy and paste the definitions from OneNote or the email back into the app.

One also paste individual set definitions in order to add them instead of specifying them using the form. This might be useful if one had a number of similar sets one wished to save, for example latest abstracts from different journals. Then one could edit the sets as desired in OneNote on a PC and then subsequently copy them over from OneNote on the phone into the webStore app. Or they can be emailed from the PC to the phone. To do this, one needs to understand the format in which the set definitions are saved.

Format for set definitions

The definition for each set consists of a few lines of information as follows:

StartURL
ShortName 
(homeDir)
(startPage)
depth
maxItems
images expireDays (lastDownloadedDate) 
numberOfIncludeStrings 
include strings on numberOfIncludeStrings lines, one string per line 
numberOfExcludeStrings 
exclude strings on numberOfExcludeStrings lines, one string per line 

The items in brackets are created when the set is downloaded so will be ignored. The other items are as follows:

Thus the definition for the BBC set might appear as follows:

http://m.bbc.co.uk/news 
BBC news 
/BBC news/index.html 
/BBC news/index.html/http/CD/m.bbc.co.uk/news/index.html 
2 
500 
1 0 
1 
uk/news 
1 
rss

Example of using set definitions to specify similar sets of pages

One way of using the set definitions is to edit them on your PC and then use OneNote to transfer them into the app. The example below provides definitions to download abstracts for papers published in different scientific journals. To obtain this list, the definition was first created for Nature Genetics by using Internet Explorer to access PubMed Mobile and providing "Nature Genetics":SOURCE as the query. Then, the URL was copied into webStore and the inclusion string /m/pubmed was added and the pages were downloaded. Then the set definitions were copied out via OneNote and edited on a PC. The definition for Nature Genetics was copied a couple of times. Then in each copy the URL and name for the set were edited. This list can now be copied back into webStore and the definitions will be added for the new journals.

http://www.ncbi.nlm.nih.gov/m/pubmed/?term=%22Nature%22%3ASOURCE 
Nature  
/Nature genetics/index.html 
/Nature genetics/index.html/shortened0//3ASOURCE/index.html 
2 
100 
0 1 20/09/2014 14:24:56 
1 
/m/pubmed 
0 

http://www.ncbi.nlm.nih.gov/m/pubmed/?term=%22Science%22%3ASOURCE 
Science 
/Nature genetics/index.html 
/Nature genetics/index.html/shortened0//3ASOURCE/index.html 
2 
100 
0 1 20/09/2014 14:24:56 
1 
/m/pubmed 
0 

http://www.ncbi.nlm.nih.gov/m/pubmed/?term=%22Nature+genetics%22%3ASOURCE 
Nature genetics 
/Nature genetics/index.html 
/Nature genetics/index.html/shortened0//3ASOURCE/index.html 
2 
100 
0 1 20/09/2014 14:24:56 
1 
/m/pubmed 
0

You can see that the homeDir and startPage are still set for Nature Genetics but these values will be ignored so there is no need to change them.

You can see a number of examples of set definitions from this page: http://www.davecurtis.net/webStore/setDefs.html

Using email instead of OneNote

Although the above methods do work with OneNote, it can actually be quite tricky marking sections to copy with OneNote on the phone. An alternative is to use email. One can copy the set definitions to the clipboard and then paste them into an email message to oneself. Then one can do all the editing one likes and email them back to the phone. Then one can copy the set definitions back from the email message more easily than from a OneNote page.