Scraping for a leading shipping company

Scraping for a leading shipping company

Customer's Need

The client, a company operating in the logistics and shipping sector, needed to monitor prices and services offered by various comparison websites and freight forwarders on a weekly basis. The main goal was to obtain detailed information on costs and services associated with different shipping parameters in order to optimize its operations and improve competitiveness in the market. Additionally, the client wanted to be promptly informed about any alerts or temporary events that could affect shipments.

Implemented Solution

Building Monitoring Bots
We developed customized bots capable of effectively monitoring numerous online portals. These bots perform data scraping to gather information on prices and services associated with shipping parameters. In particular, we acquired details regarding pickup and delivery mode, available value-added services (VAS), delivery and pickup times, thus providing the client with a comprehensive overview of available options in the market.

Detection of Temporary Alerts
We implemented a system to detect and monitor any temporary alerts communicated on target sites. This included alerts related to ongoing strikes, service suspension due to force majeure events, and other situations that could affect shipments. This feature allowed the client to be promptly informed about any changes or interruptions in shipping services.

Scalable Infrastructure
We created a highly scalable infrastructure that allows the client to dynamically adjust monitored routes and increase monitoring volumes based on needs. This provided the client with the necessary flexibility to quickly adapt to changes in the market and optimize its shipping strategies.

Advanced Reporting
We developed an advanced reporting system that allows the client to assess its market positioning based on various filters, such as prices, services offered, and other relevant parameters. The reports are fully customizable to meet specific needs and preferences, providing a detailed and personalized analysis of performance in the logistics and shipping sector.

The implemented solution provided the client with greater visibility on pricing and services, allowing for optimized operations, improved competitiveness, and timely alerts regarding potential disruptions in shipments.

The client faced challenges such as a constantly evolving market and the need to consider numerous factors, including the year of production, brand, model, and vehicle conditions, in determining the optimal price for its vehicles.

A solution involving crawling and data collection from target sites was implemented, along with the construction of custom bots for monitoring, detection of temporary alerts, modellable infrastructure, and advanced reporting.
Contact us for help

Get in touch and let us know how we can help touch as soon as possible.

Contact Us