In years past, spying on your competitors was certainly something many companies did, but it was a manual, labor-intensive process. For example: a “mystery shopper” would walk through a retail store and note the prices being charged for various items to be sure their own employer’s store was priced competitively. Now, thanks to the speed of online transactions and the thoroughness of scripted bots, retailers from Amazon to Walmart and even many small startups are replacing the antiquated human “mystery shoppers” with powerful web page scraping tools and hardcore analytic software.
According to Alexandr Galkin, CEO of Competera, companies including Amazon and Walmart have entire teams dedicated to scraping. Competera itself admittedly scrapes pricing data from across the web for retailers, and uses leading edge machine-learning algorithms to help its customers decide how much to charge for their products.
According to Wired magazine, “Some of Competera’s customers, including Acer Europe and Panasonic, use the company’s ‘brand intelligence’ service to see what retailers are charging for their products, to ensure that they are complying with pricing agreements.”
What is starting to come into question are the privacy rights of companies verses those of individuals. Many commercial sites are now deeply familiar with the privacy restrictions of GDPR legislation. However, those restrictions are extended to corporate identities and data- yet.
With US Law characterizing corporations as people thanks to an erroneous dictum from the Santa Clara County v. Southern Pacific Railroad case of the 1880s, one could easily see room for an argument that pricing data is just as worth of privacy protection from others as the personal information of anyone working at the company itself.
Right now, a technical battle of bots, captcha, and anti-scraping scripts is being waged on many of the largest retail sites in the world. Will that war soon be waged in courtrooms as well? The goal of an Internet that is ubiquitously available is now facing the same sorts of constraints that offline personal privacy and commercial competition faced before digital media was invented.
NationalNet hosts commercial retail accounts- large and small- for our customers. We work hard to protect your data and are available to work with you on any efforts you may be undertaking in this regard. As the Internet continues to evolve, having trusted technology partners on the backend is sure to make your front end operations far simpler and more successful over the long term.