Everything from what new item is selling well on eBay, to what "Wired Magazine" is saying about latest technology. More and more magazine and newspaper publishers are gaining a foothold on this new juggernaut which is going to change way we gather news.

Advertisers too are not left out. Already some of largest media companies in world are now poised to find advertising space for their clients 'branding' campaigns, on any available web page that regularly features in rss feeds.

The media companies with billions of ad dollars to spend, will they say, be spending less on newspaper and t.v. advertising in coming years, and will concentrate instead on 'blogs' and rss feeds.

Competition will be fierce for every available space on most popular feeds, some run by newspapers!

Many freelance journalists will be running rss feeds in direct competition with newspaper employers who now pay them to gather news for their publication. There will be no editor to censor their news items, they will be free to write whatever they want.

In this article, I'll tell how you can use Robogen to simplify process of creating a robots.txt file. This article assumes that you have read my article on what a robots.txt file is and how you can manually create it. If you missed article, it is available at http://www.1stSearchRanking.com/t.cgi?4901&robots.htm

I would recommend that you download trial version of Standard Edition. The Limited edition does not support as many search engines as Standard Edition.

Once you have downloaded Robogen and installed it in your hard drive, start program. Here are steps that you should follow to create a robots.txt file:

Step 1: Click on Logon FTP Server button. In dialog box that appears, type in address of FTP server for your web site, make sure that "This server requires me to log in" option is checked, and then type in username and password which you use to access server in appropriate boxes.

Now, depending on how your web host has set up server, files for your web site may be present in root directory of server itself. Or, it may be present in a sub-directory of root directory. In latter case, click on "Use specified root" option and type relevant sub-directory in text box besides option. If you don't know in which sub-directory files for your web site are present, ask your web host to give it to you.