Smabbler | SmabblerBot | FAQ
Welcome to Smabbler!
Smabbler has been working towards creation of a World Wide Web business research engine based on the concepts of applying intelligent machine learning techniques for reading unstructured data (such as text designed to be read by humans) published on the web and transforming them into searchable information and business insight.
The project was founded in late 2014 and ever since has been expanding its resource data base to allow businesses and entrepreneurs to find and explore new cooperation and partnership opportunities.
What is it really about?
If you read this then you must be online and will probably agree that the Internet has become ever so important in our everyday activities, whether personal or business related.
Web pages are built using text-based mark-up languages and frequently contain a wealth of useful data in text form. However, most web texts are designed for human end-users and not for ease of automated understanding, summarization and use.
This is especially significant for various enterprises which rely heavily on the web for finding others and to be found by others. At the same time, not many tools are dedicated and tailored specifically for those businesses’ needs. This is where smabbler comes in helping, through text processing and semantic understanding, to find information and relevant leads quickly and efficiently.
SmabblerBot is an Internet bot utilizing a universal software technique adopted by search engines for the systematic browsing of the World Wide Web for the purpose of Web indexing, information extracting, and web data integration.
Our Web search engine uses Web crawling software to update the search content and indexes of others sites' web content. Our Web bot scans pages it visits for later processing by the search engine which indexes the downloaded information and transforms unstructured data on the web into structured data, so the users can search for the site much more efficiently.
Internet bots can retrieve data much quicker and in greater depth than human searchers, so they can have an impact on the performance of a site. Our bot minimizes resources spent fetching pages by focusing primarily on related and similar topics and carefully prioritizing and managing the hyperlink exploration process. SmabblerBot explicitly does not scan and omits large web files and limits itself to access intervals and the number of urls visited per domain.
At the same time, SmabblerBot abides by the robots exclusion protocol, also known as the robots.txt protocol that is a standard for administrators to indicate which parts of their Web servers should not be accessed by crawlers.
The Web has a very dynamic nature, and scanning a fraction of the Web can take weeks or months. By the time an Internet bot has finished its scan cycle, many events could have happened, including creations, updates and deletions. Additionally webmasters are able to use an extra "Crawl-delay:" parameter in the robots.txt file to indicate the number of seconds to delay between requests to include a suggestion for the interval of visits to the same server.
While most of the website owners are keen to have their pages indexed as broadly as possible to have strong presence in search engines, it is always good and highly recommended to have a web robot management system in place.
Apart from standard web application security recommendations website owners can reduce their exposure to opportunistic no-good doers by only allowing (with robots.txt) search engines to index the public parts of their websites and explicitly blocking indexing of transactional parts (login pages, private pages etc.).
Our Web bot identifies itself to a Web server through the User-agent field of an HTTP request. Web site administrators can examine their Web servers' log and use the user agent field to determine which Internet bots have visited the web server and how often. The user agent field include a URL linking to this site where the Web site administrator may find out more information about the bot. SmabblerBot’s user agent is: SmabblerBot/1.0 (https://smabbler.com/en/Home/About)
1. How can I make my business website show up in Smabbler?
Smabbler indexes sites and pages which reach the requirement of user search experience. The value of pages determined by their topic and detailed content is the reason justifying the indexing by Smabbler. To help Smabbler discover your site more quickly, your are welcomed to submit your website address link here:
2. How can I create my company profile if it has no website?
We are sorry, but Smabbler currently does not support manual company profile data input. We plan to initiate such an option in the near future, so please sign in to be informed about new updates and features from Smabbler. In the meantime, you are welcome to search and discover other companies using our service.
3. Why doesn’t my page appear in the Smabbler search results?
Smabbler does not promise that all pages can be searched. If your page fails to be searched by Smabbler for a considerable time or suddenly disappears from the search results, it might mean that your website does not fit the user search experience, or we encountered technical problems while attempting to index your site.
4. How to prevent my website from being indexed by Smabbler?
Our Internet bot strictly complies with robots.txt protocol. For detailed information, please visit http://www.robottxt.org/. We will not scan sites with appropriate robots.txt protocols. Please note, the index requires several months to be removed from the search engine database. For an urgent request to remove your site, please visit: