"I want to collect data from this site, but it isn't in the bot list."
This is the wall people most often run into when they want to outsource crawling. Writing code takes time, while outsourcing means getting a quote even for a small task. So Hashscraper has opened a beta feature for creating crawling bots with AI. Describe the site you want to collect from and the data you want in plain language, and AI creates a collection method, collects a sample to show you, and runs it as a bot after human review. This article shows, in conversation order, the actual process of creating a bot for a blog.
Definition. No-code crawling is a way to collect data from web pages without writing code. There are methods where you click and select items on the screen, and methods where AI creates a collection method based on a description of what you want.
Definition. Hashscraper's "Create a Bot with AI (Beta)" is a feature that creates a collection method (bot) through conversation, then collects data on Hashscraper servers after safety checks and administrator review. The collected data is provided as a dataset.
[Create a Bot with AI (Beta) screen in the left menu — New Bot button and list of created bots](images/040/00_AI로봇만들기_메뉴.png)
▲ Click "New Bot" under "Create a Bot with AI (Beta)" in the menu to open a conversation with AI.
TL;DR
- Enter an address and describe the data you want, and AI creates a bot that handles list → post details → multiple pages.
- Using Hashscraper CEO Kyungho Kim's personal blog (bamchi.me) as an example, we created a bot in just a few exchanges and collected 27 posts in about 1 minute 20 seconds (title, date, views, body, and URL).
- Creation and sample collection are free, while collection costs a base rate of 25 credits per item. Pages requiring login, CAPTCHAs, apps, and personal information cannot be used.
1. Start With a Single Address
This was all we sent at first.
Collect blog posts from Bamchi.me
AI first opened the page itself. It then replied: "There are 9 posts per page, divided across multiple pages. The list includes each post's title, date, summary, and post URL."
The important point here is that no one explained the HTML structure. AI read the page and figured out which tags contained titles and how the next-page URL changed.
[Conversation with AI — after sending a single address, AI opens the page and asks whether to collect only the list or include post bodies](images/040/01_대화주소와방식.png)
▲ The actual conversation screen. After receiving a single address, AI opens the page and first asks what to collect and how.
2. List Only? Include the Full Body? — AI Asks First
Rather than creating the bot immediately, AI asked how we wanted to proceed.
- Collect the list only — Quickly collect titles, dates, summaries, and URLs
- Collect full post bodies — Collect post URLs from the list, then open each post and retrieve the complete body
After choosing option 2, it asked another question: "How many pages would you like to collect? Three pages would be about 27 posts. I can also make the page count adjustable each time you collect." After asking it to do so, "number of pages to collect" became a bot setting. You can use 1 page this week and 10 pages next month without rebuilding the bot.
Two-stage crawling from a list to detail pages, along with pagination, is usually the most labor-intensive part of coding. Here, the process was completed with two questions.
3. Preview It With a Sample Collection
AI collected 5 posts from the first page as a test and showed the results.
| Title | Date | Views |
|---|---|---|
| GPT-5.6 Sol, the Trap of the Most Powerful Model | July 13, 2026 | 2,652 |
| What People With Money and Time Buy First — It Is Neither Cars nor Luxury Goods | February 17, 2026 | 5,028 |
| The Story of Building My Own AI Assistant With a Mac Mini | February 2, 2026 | 4,708 |
It also included view counts, which were not requested. AI saw that the post pages contained view counts and suggested including them. The sample collection also verifies whether the next page actually opens.
[AI message showing sample collection results and instructions to request review after saving](images/040/02_대화시범수집저장.png)
▲ Sample collection results are shown in the conversation, and the collection method is saved when you reply "Save."
[Final sample collection panel — finds 9 URLs in the list, opens 5 detail pages, and previews 5 rows](images/040/04_시범수집_미리보기.png)
▲ A sample collection table also appears on the right side of the screen. It even verifies that 9 posts appear on the next page.
If you are not satisfied with the result, you can revise it in plain language right there. For example: "Remove the date," or "Show only the first 200 characters instead of the body."
4. It Runs Only After Human Review
After saving, clicking "Request Admin Review" does not start collection immediately. It goes through two stages.
- Safety check (1–3 minutes) — Checks whether the URLs are safe, whether the site's collection rules (robots.txt) block access, whether the sample results contain values that appear to be personal information, and whether there are too many requests at once. This bot passed with "approximately 30 requests at a time, at 1-second intervals per site."
- Administrator review (usually 1 business day) — A person reviews the collection method and the credits required per item before approving it.
[Bot being created panel — collect URLs from the list → extract values from details, Number of Pages to Collect input, approved · 25 credits per item](images/040/03_만든봇승인.png)
▲ Once approved, you can see the bot's collection method, input fields, and per-item credits at a glance, then continue with "Open Bot."
There is a simple reason why a person reviews the bot once more rather than running exactly what AI created: to prevent bots that burden someone else's site or collect data that should not be collected. This example was the CEO's own blog, so there was no issue, but the member who creates the bot is responsible for complying with the target site's terms of use and applicable laws.
5. Collect With the Bot — 27 Items, 675 Credits
After approval, we entered 3 for the page count on the bot screen and started collection.
[Bot screen — Start Collection button and collection result of 27 items, 675 credits](images/040/05_봇화면수집결과.png)
▲ Bot screen. Start collection and view results from the same interface used for other crawling bots.
- Collection time: about 1 minute 20 seconds
- Result: 27 posts (title, date, views, body, and post URL)
- Cost: 25 credits per item × 27 items = 675 credits
Just like data collected by other bots, the collected data is accumulated in a dataset and can be downloaded as an Excel file.
[Dataset screen — 27 collected items, 675 credits used, table with title, date, views, body, and page URL](images/040/06_데이터셋_27건.png)
▲ The 27 collected posts have been added to the dataset. You can download them with "Download Excel."
What You Can and Cannot Create With AI
| You can create | You cannot create |
|---|---|
| Public lists, boards, and product pages | Pages visible only after logging in |
| Two-stage list → detail crawling, multiple pages | Pages that display robot verification (CAPTCHA) |
| "Load more" buttons and pages rendered only after opening the screen (collected through a browser, so it takes longer) | Apps, personal information |
For heavily blocked shopping malls or sites with logins and CAPTCHAs, it is better to submit a development request for a Hashscraper developer to build the collector directly instead of using an AI bot.
Frequently Asked Questions
Q. Does it cost anything to create one?
Creation and sample collection are free. Only actually collected data is charged at a base rate of 25 credits per item. If the rate changes after administrator review, you will be notified.
Q. Do I need to know any code at all?
No. You only need to describe the address and the data you want in plain language. For example: "I want to search for shampoo on an online store and collect product names, prices, and review counts."
Q. Where does the conversation content go?
It is sent to an external AI service provider in the United States to create and review the bot. For this reason, users are advised not to include personal information or passwords in the conversation.
Conclusion
Until now, creating a crawling bot was a development task involving "reading the page structure, choosing selectors, and building pagination." Creating bots with AI turns this work into a conversation. Start with a single address, answer two questions, review the sample results, and collect with a bot that has been reviewed by a person. It is still in beta, so there may be shortcomings. Please try it and let us know, and we will improve it quickly.
Get Started Now
- Create one yourself: Create a Bot with AI (Beta)
- For sites with logins or blocking: Contact Us About Crawling
