Seeing the Invisible Web

You are probably wondering what on earth the invisible web is. Is it some concept to come or is it something that already exists? To understand, we need to look at what you already know, the visible web. When we talk about the visible web, you are probably aware that we mean the part of the web that is visible by search engines and that is accessible to the general public. Now we are getting somewhere arent we? You have probably guessed what is the invisible web now ha? Yes, this is the part of the web that these search tools cannot find. Todays invisible web however is very different to what it was just a few years ago.

For example, in the year 2000, very little progress had been made in what todays indexing programs are capable of. Thus, the invisible web of year 2000 has literally become visible, thanks to these developments. Non-HTML page formats, like PDF, PowerPoint, word and excel spreadsheets were invisible back then, but today, they are as visible as the noon-day sun. Think also of the script-based pages and dynamically-generated pages created using database software like Active Server Pages (ASP) and Cold Fusion. All these pages are now indexed by most search robots, as long they have an extended presence online.

That however does not mean that the web is completely visible. As new robotic ways of searching online content has evolved, new software and new formats have also found their way on the web and as a result, some hurdles still remain to be leaped. For example, some contents of searchable databases, especially those that require login authorization or keywords in a certain search box remain hidden from view.

How can you access the invisible web? One way is to be on the look out for databases and contents from authorized users. Most of this content may only be viewed by those who have search authorization. Be sure to subscribe to these vast library resources to tap on the vast amount of information hidden in these places. Another way to access the invisible web is to search specifically for database using the word database. If, for example, you are searching for cars companies in California, the search keyword should read cars in California database. You will be surprised by the databases that will come up that are completely invisible to average users.