Hash scraper technology blog

How do you collect and curate LLM fine-tuning datasets from the web? The winners are not the teams that scrape the most, but the ones that filter the best.

How do you collect and curate LLM fine-tuning datasets from the web? The winners are not the teams that scrape the most, but the ones that filter the best.

The first thing a team that decides to build its own sLLM usually does is "crawl." They scrape millions of pages, boast about the volume of text, and declare, "Corpus secured." Then, the moment the...

Read more →
In the age of AI answers, is our brand being cited by ChatGPT and Gemini? — “If you’re not on the list, it’s not that you rank low.”

In the age of AI answers, is our brand being cited by ChatGPT and Gemini? — “If you’re not on the list, it’s not that you rank low.”

"Recommend a place that does ○○ well." What customers used to type into a search bar, they now ask ChatGPT. The response listed three places. We were not among them. With search, even if we ranked ...

Read more →
When it comes to web data collection for the first time, there is only one choice that cannot be undone

When it comes to web data collection for the first time, there is only one choice that cannot be undone

You've probably read about ten comparison articles. But you still haven't made any decisions. It's not because of lack of information. It's because you believe that once you make the wrong choice, ...

Read more →
Integrating Crawled Data into a Data Warehouse — Connecting External Web Collection to the Pipeline

Integrating Crawled Data into a Data Warehouse — Connecting External Web Collection to the Pipeline

When “We Want to See Competitor Data on Our Dashboard Too” Reaches the Data Team Internal sources are stable. We control the schema, changes are announced through release notes, and there are repro...

Read more →
Why Does External Web Data Keep Breaking? — Data Contracts in the Era of Website Redesigns

Why Does External Web Data Keep Breaking? — Data Contracts in the Era of Website Redesigns

The Pipeline That Was Fine Until Yesterday Is Empty This Morning This rarely happens with internal sources. Changing a schema requires alignment, removing a column comes with advance notice, and wh...

Read more →
What service should I use for real-time extraction of large-scale web data? — First, answer "How many minutes?"

What service should I use for real-time extraction of large-scale web data? — First, answer "How many minutes?"

"Please collect in bulk and in real time." This is the most common sentence in collection inquiries. And this sentence does not provide any information to estimate. The pilot probably went well. A ...

Read more →
How to choose a data collection service for e-commerce price comparison and monitoring? It all depends on the question, "Can it cover Coupang?"

How to choose a data collection service for e-commerce price comparison and monitoring? It all depends on the question, "Can it cover Coupang?"

"Do Coupang and Naver Shopping get collected as well?" People who are looking into price collection services start asking this question from the age of nineteen. The problem is that this question d...

Read more →
How to monitor investment research and risks through news, disclosures, and community crawling.

How to monitor investment research and risks through news, disclosures, and community crawling.

Can one analyst see the opinions of hundreds of stocks? Research organizations typically cover dozens to hundreds of stocks. With dozens of news articles pouring in for each stock every day, along ...

Read more →
Analyzing property listings and price trends through real estate platform crawling.

Analyzing property listings and price trends through real estate platform crawling.

Are you still compiling local and site-specific market prices manually? Developers, intermediaries, and construction material suppliers often encounter moments when they need to determine whether t...

Read more →

Get notified of new posts

We'll email you when 해시스크래퍼 기술 블로그 publishes new content.

Your email will only be used for new post notifications.