71% finance organisations use web scraping to gather external data

In cooperation with Censuswide, Oxylabs surveyed 500 UK-based and 501 US-based senior data decision-makers from UK and US financial services companies, gathering insights on attitudes towards data collection in the sector.

Internal data (74%) remains the most popular source for businesses, but there are clear signs web scraping is set to overtake it soon. Over two-thirds of respondents (68%) plan to increase their data budgets in the coming year, while 80 per cent believe they will focus more strongly on web scraping in the same period. Other methods, such as third-party data aggregators (used by 56%) remain an important part of the data-gathering ecosystem but are now a long way behind web scraping and internal data.

Key highlights:

  • 71% of respondents use external data gathered using web scraping to guide their business decisions;
  • Web scraping follows closely behind internal data (74%), with the former’s adoption on an upward trajectory;
  • Web scraping and internal data rank well ahead of other data-gathering methods, with third-party aggregators being the third-most-popular option (56%);
  • 80% believe their focus will shift more heavily towards web scraping in the next 12 months, while 68% plan to increase their data budgets during the same period;
  • 38% say data risk and legal complications are their main concerns around adopting web scraping, while 36% cite complicated tech and budget constraints, and 34% a lack of technical expertise.
  • Almost three-quarters (71%) of financial services organisations are now using web scraping to gather external data to guide their decision-making.

Gediminas Rickevičius, VP of Global Partnerships at Oxylabs said, “internal data is relatively cheap, efficient, and company-unique, so it’s not a surprise that it’s the dominant acquisition method. Alternative data and web scraping being so close, however, is something completely new. There’s a clear interest and even necessity to uncover new ways of unearthing data-driven insights from untapped sources. It’s clear – financial services companies are expecting to get their edge through web-scraped alternative data.”

Despite web scraping’s rise, there remain some concerns among decision-makers about how to implement it in a safe, cost-effective, and compliant manner. Almost four in ten (38%) state risk and legal complications as the main factors discouraging them from using web scraping. Alongside this, 36 per cent are concerned about budget constraints or the need to introduce complicated new technology, and 34 per cent believe they lack the technical know-how to make web scraping a success.

Vaidotas Šedys, Head of Risk Management at Oxylabs added, “risk is inherent to all new technologies, as pioneering a new field means there are no footsteps to follow. Any web scraping risks, however, can be greatly mitigated by applying data collection and management best practices and employing legal and risk professionals to ensure web scraping is done in a compliant, safe, and ethical way.

“These practices may take some time to get used to, but done in the right way, there’s every chance that financial services businesses can gain valuable new insights, improve customer experiences and achieve strong ROI through web scraping.”

More News
1 day ago
Canon aims to inspire large format graphics community at FESPA 2022
Canon will be helping print service providers (PSPs) and print factories to ‘Make It BIG’ by being bolder and leaving a more lasting impression on their customers. The ‘Make It BIG’ campaign has been developed for this year’s event as a continuation of Canon’s ‘Make It’ drive to help print professionals boost business growth, optimise performance, and maximise new opportunities in large format graphics.
1 day ago
Sharp introduces Synappx Collaboration Hub for hybrid meetings
Sharp has announced the forthcoming launch of Synappx Collaboration Hub, a smart subscription service that aims to simplify the meeting experience, addressing the changing needs and challenges of hybrid meetings.
2 days ago
Harrods achieves total paper towel sustainability with Banner
Luxury department store, Harrods has diverted tonnes of its used paper hand towels from general waste.
4 days ago
Marsden Building Society chooses Avaya Cloud Office
Avaya, experts in solutions to enhance and simplify communications and collaboration, is providing Marsden, Lancashire’s largest building society, with its Avaya OneCloud UCaaS solution for more agile, flexible, and scalable communication.
4 days ago
New DURABLE RISE to tackle remote working ergonomics
DURABLE is launching a solution in the form of the laptop stand RISE. The stand ensures that the device is at the perfect height, it also makes it possible to work ergonomically with an external keyboard and mouse or a second monitor. The infinitely height-adjustable stand is available now.
4 days ago
Kodak Alaris enhances i4000 Series Scanners
Kodak Alaris has enhanced its Kodak i4000 Series Scanners. The Kodak i4650 and Kodak i4850 Production Scanners now deliver faster scanning speeds of up to 160 pages per minute (ppm).
5 days ago
Fax remains a pervasive technology in the office
Fax remains a pervasive technology, powering many of the things we all rely on from finance and insurance, to buying a house, and healthcare, and forgetting the food on our plates. This is according to the latest research from eFax, the cloud-based fax solution for businesses.
1 week ago
Getting data right key to digital transformation
Data is the key to transforming any business. Accelerated by COVID-19, businesses have started to become more data-driven and embrace digital transformation strategies. Failure to understand data, what a business has, and what a business needs, is compromising far too many digital transformation plans, and leading businesses to waste years on projects that ultimately, will never deliver.
1 week ago
Cheetah Digital helps football leaders win loyal fans with data
Recent UK research suggests that data will now have a more important role in fan engagement than ever before. Sports clubs are now re-evaluating how to future-proof their strategy to attract and retain loyal fans.
1 week ago
Dynabook extends Satellite Pro C40 range with AMD Ryzen processors
Dynabook Europe has announced the expansion of its Satellite Pro C40 range to include AMD Ryzen 5 and 7 series processors with Radeon Graphics, broadening its range of budget-conscious business laptops.

Login / Sign up