What should I prepare before making an inquiry?
The most common hesitation for those considering outsourcing web scraping for the first time is surprisingly the inquiry itself. They often wonder, "How do I explain when I don't know the technology?" or "What should I decide on before contacting?" This can delay the review process for weeks.
In conclusion, you don't need to prepare perfectly for an inquiry. The requirements are usually sorted out together after the inquiry, and that process itself is part of the onboarding process. However, knowing how the overall process flows from the inquiry to receiving the initial data will speed up each step. This guide is a step-by-step overview from making an inquiry to receiving the first data.
7 Steps of Onboarding Process
1. Inquiry
This stage involves informing us of what data you want to collect and how you plan to use it. A simple sentence like "I want to monitor competitor prices" is sufficient. It's not necessary to have specific target sites or items decided at this point.
2. Requirement Definition
Based on the inquiry, we specify the target sites for collection, the data fields to be collected, the scale and frequency of collection, and the delivery format. If you let us know your objective, we may suggest, "For that purpose, you need this item from this site," in return.
3. Feasibility Check
This stage involves actually checking the defined target sites. We assess the feasibility and difficulty of collection by examining factors such as blocking intensity, login requirements, and whether the desired items exist on the screen. Adjustments like "This item is not available on this site" or "Another source is better for this channel" may occur here.
4. Estimate
Based on the feasibility check results, we provide you with an estimate. The estimate is free, and we also guide you on whether individual billing or subscription is more suitable at this stage. If you're curious about the cost structure, reading the estimate and TCO comparison articles first will expedite the conversation.
5. Contract
Finalize the pricing plan, duration, and delivery conditions. Three things to confirm at this stage are whether additional costs will be incurred for adding sites, whether maintenance is included, and whether mid-term adjustments or cancellations are possible.
6. Onboarding (Initial Setup and Inspection)
This stage involves setting up the crawler and inspecting the initial collection results as a sample. We check if the fields are captured as intended, if the value format is user-friendly, and make adjustments as needed. The more thorough the inspection, the smoother the operation will be in the future.
7. Regular Delivery and Operation
Data starts arriving at the confirmed frequency. We handle crawler operation, maintenance, and monitoring, and if a site changes causing disruptions in collection, we detect and respond to it first.
Things to Prepare Before Making an Inquiry
Although not mandatory, having the following prepared will significantly shorten the requirement definition (2nd step).
- Objective: What decision you want to make with this data (most important)
- Target Sites: A few candidate URLs (does not need to be finalized)
- Data Fields: Required information (e.g., product name, price, review text, rating)
- Scale and Frequency: Approximate keywords, number of products, daily/weekly, etc.
- Delivery Format: Preference among Excel, email, API, DB integration, dashboard
- Future Plans: Possibility of expanding sites/items
Inquiries are possible even if all fields are empty. However, having a clear objective like the first line (Objective) is beneficial. The clearer the objective, the easier it is for us to fill in the rest.
What to Look for in Feasibility Check
Since the 3rd step significantly affects the quality of the onboarding process, let's explain a bit more. Three things to check at this stage are:
- Accessibility: The blocking intensity and login barriers of the target site. Sites with strong blocking require corresponding infrastructure, affecting the difficulty and cost.
- Presence of Items: Whether the desired information actually exists on the screen. Items that are not present cannot be collected by any means, so at this stage, we find alternative sources or make adjustments to items.
- Structural Stability: How robust the page structure is for collection. Sites that undergo frequent redesigns pose a maintenance burden, influencing the choice of billing method (subscription or not).
If a company skips this check and proceeds directly to the contract, caution is advised. Discovering that data cannot be collected after the contract is a loss in terms of both time and cost.
After Operation Begins: How are Additional Requests Handled?
For individual billing, adding sites/items during operation is a new order — it incurs additional costs for estimation, approval, and development.
With a subscription model, this process is reduced to a single request. Additional development is included in the monthly fee, so if you say, "Please add this site as well," it will be implemented without a separate estimate. This means you don't have to go through steps 1-5 of the onboarding process again, and the gap between the need for data and the start of collection shortens.
Frequently Asked Questions
Q. Do I need to know any technology at all?
No, you don't. All you need is to know "what you want to do with what data." Site structure and collection technology are areas we will handle.
Q. How long does it take until onboarding?
It depends on how clear the requirements are. If the targets and items are decided, the review and estimate process will be faster. Starting together from requirement definition will extend the earlier steps. If you inform us of any schedule requirements at the inquiry stage, we will guide you accordingly.
Q. Can I make a decision after seeing the data?
During the onboarding stage, we inspect the initial collection results as a sample and adjust the fields and formats together. Before the contract, we guide you on how each item will appear based on the feasibility check results.
Recommended Articles to Read Together
- Web Scraping Estimates, Why Do They Differ Among Companies — Cost structure and hidden costs
- Web Scraping Subscription vs. Individual Billing, the Answer Lies in the Total Cost of Ownership (TCO)
- How to Choose a Web Scraping Company — 7 things to check before outsourcing data collection
Start Now
You can start with just one line of objective. We will help you with the rest of the steps.
