The digital workspace is flooded with numerous data sources, and searching for information across these platforms becomes a monumental task for businesses. Recognizing this pain point, Amazon Kendra has announced the release of its Web Crawler. This new feature primarily targets enabling businesses to search content across their websites seamlessly.
This article dives deep into understanding how the Amazon Kendra Web Crawler enables indexing web pages made with Angular, JavaScript or React, and offers intelligent search capabilities across such content. The guide also highlights functioning with VPC support that enables indexing content from internal websites which are accessible through a VPC subnet that one can configure.
Table of Contents¶
- Introduction to Amazon Kendra’s Web Crawler
- Web Crawler’s Key Functions
- How Does Amazon Kendra’s Web Crawler Work?
- VPC Support and Its Advantages
- Setting Up the Web Crawler
- Driving Business Value With Intelligent Search
- The Place of Amazon Kendra in the Market
- Final Words
Introduction to Amazon Kendra’s Web Crawler ¶
Amazon Kendra is a highly accurate and user-friendly search service powered by machine learning. It utilizes natural language processing to field users’ questions, read documents, and extract answers. The recent introduction of the Amazon Kendra Web Crawler takes these capabilities to new heights. It provides a new level of intelligent search that can index HTML content from intranet sites or public websites directly.
Web Crawler’s Key Functions ¶
The Kendra Web Crawler performs several functions:
- Indexing: It indexes websites built with diverse JavaScript frameworks such as Angular, React, or vanilla JavaScript.
- Intelligent Search: The crawled data enables intelligent search across the indexed content.
- Support: It provides VPC support to index internal websites accessible through a pre-configured VPC subnet.
The result? You get top-notch search performance to find relevant data regardless of source or content complexity, without needing a developer to write a custom indexer.
How Does Amazon Kendra’s Web Crawler Work¶
The web crawler uses an automatic scheduling feature where it seeks out and indexes new web pages or updates to existing ones at regular intervals. It provides the flexibility to set crawling schedules as per your business needs.
To enable the indexing of dynamic web pages, the Web Crawler uses a headless browser, a web browser without a user interface. This enables it to find and index content with JavaScript-based front-end frameworks by rendering the page and executing the JavaScript, just as a standard browser would.
VPC Support and its Advantages ¶
Kendra’s Web Crawler comes with optional VPC support. This feature is particularly beneficial for businesses with private data that can only be accessed within their VPC. Connecting Amazon Kendra to your existing AWS VPC, businesses can crawl and index content from their internal websites or documents stored privately in their VPCs.
Setting Up the Web Crawler ¶
To actively use Amazon Kendra’s Web Crawler, you need to configure the data sources you would like to be crawled. Once set, the Web Crawler progressively starts to index the pages and records all successful web crawls.
Driving Business Value With Intelligent Search ¶
Keeping pace with today’s digital landscape, businesses should generate business intelligence from the vast data pools they curate. Companies can tap into the potential of their internal and external data with Amazon Kendra’s Web Crawler’s dynamic information search capabilities.
The Place of Amazon Kendra in the Market ¶
The market for intelligent search is indeed crowded, but Amazon Kendra stands out for its comparative strength in indexing and search capabilities, thanks to features like the Web Crawler. As businesses continue to grapple with data spread across multiple sources, tools such as Amazon Kendra become more valuable.
Final Words ¶
Amazon Kendra’s Web Crawler offers dynamic content support to businesses by streamlining the indexing and search process of HTML web pages. Its intelligent search capability presents businesses with precise results stripped of data complexity. The journey towards efficient data search and analysis has indeed been empowered with the launch of the Kendra Web Crawler.